2024-05-08T23:16:38.798 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T23:16:38.799 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175 -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/orig.config.yaml 2024-05-08T23:16:38.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:16:38.950 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698175' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - overall HEALTH_ - \(MON_DOWN\) - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu9zWuOI0lYhRv2seQduy2OCDHF5sybPrgIHTaBs9ah3XxHHZrIWwwtTYbii8CQZeuNlPU5zhms4XBe3hKyXDY= smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFny0qGZEQajeVqgn29XdrGECbC5fjbeuf9BUOXV7JZuevDcuFXUbrzfTvH2dPI1Y5cOfVojETcNqNX/otdtiOo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T23:16:38.950 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T23:16:38.951 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T23:16:38.951 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T23:16:38.953 INFO:teuthology.task.internal:Checking packages... 2024-05-08T23:16:38.976 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T23:16:38.977 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T23:16:38.977 INFO:teuthology.packaging:ref: None 2024-05-08T23:16:38.977 INFO:teuthology.packaging:tag: None 2024-05-08T23:16:38.977 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:16:38.977 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:16:38.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:16:39.124 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T23:16:39.126 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T23:16:39.137 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T23:16:39.137 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T23:16:39.178 INFO:teuthology.task.internal:Saving configuration 2024-05-08T23:16:39.197 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T23:16:39.209 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T23:16:39.235 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:11:53.139900', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu9zWuOI0lYhRv2seQduy2OCDHF5sybPrgIHTaBs9ah3XxHHZrIWwwtTYbii8CQZeuNlPU5zhms4XBe3hKyXDY='} 2024-05-08T23:16:39.257 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 23:11:53.141151', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFny0qGZEQajeVqgn29XdrGECbC5fjbeuf9BUOXV7JZuevDcuFXUbrzfTvH2dPI1Y5cOfVojETcNqNX/otdtiOo='} 2024-05-08T23:16:39.257 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T23:16:39.268 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-05-08T23:16:39.268 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-05-08T23:16:39.268 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T23:16:39.375 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f32f30b4160>, signals=[15]) 2024-05-08T23:16:39.375 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T23:16:39.387 INFO:teuthology.task.internal:Opening connections... 2024-05-08T23:16:39.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:16:39.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:16:39.466 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:16:39.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:16:39.538 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T23:16:39.544 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-05-08T23:16:39.566 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-05-08T23:16:39.566 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel" 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora" 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.6" 2024-05-08T23:16:39.627 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:16:39.628 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:16:39.629 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-05-08T23:16:39.662 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-05-08T23:16:39.684 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-05-08T23:16:39.685 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-05-08T23:16:39.745 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:ID="rhel" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="fedora" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8.6" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T23:16:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:16:39.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T23:16:39.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T23:16:39.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T23:16:39.747 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T23:16:39.747 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-05-08T23:16:39.769 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T23:16:39.784 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T23:16:39.805 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T23:16:39.806 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:16:39.808 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-05-08T23:16:39.828 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T23:16:39.839 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T23:16:39.839 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:16:39.869 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-05-08T23:16:39.904 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T23:16:39.960 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T23:16:39.978 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-08T23:16:39.978 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T23:16:39.978 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T23:16:39.979 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:16:39.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:16:39.999 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-08T23:16:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:16:40.061 INFO:teuthology.task.kernel:Running kernel on smithi033: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:16:40.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-05-08T23:16:41.677 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T23:16:41.677 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T23:16:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:41.910 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:16:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:43.673 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-05-08T23:16:44.726 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2024-05-08T23:16:48.815 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 417 kB/s | 24 kB 00:00 2024-05-08T23:16:49.626 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T23:16:49.626 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T23:16:49.717 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T23:16:49.718 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-08T23:16:49.718 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T23:16:49.862 DEBUG:teuthology.orchestra.run.smithi033:> echo no | sudo yum reinstall kernel || true 2024-05-08T23:16:50.250 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T23:16:50.250 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T23:16:50.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:50.531 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:16:50.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:50.910 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 11:16:48 PM UTC. 2024-05-08T23:16:50.993 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:16:50.994 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:16:51.032 INFO:teuthology.orchestra.run.smithi033.stderr:Error: No packages marked for reinstall. 2024-05-08T23:16:51.088 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum reinstall -y kernel || true 2024-05-08T23:16:51.471 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T23:16:51.471 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T23:16:51.703 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:51.703 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T23:16:51.703 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:16:52.060 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:04 ago on Wed 08 May 2024 11:16:48 PM UTC. 2024-05-08T23:16:52.129 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T23:16:52.129 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T23:16:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:Error: No packages marked for reinstall. 2024-05-08T23:16:52.216 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T23:16:52.390 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:16:52.390 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T23:16:52.391 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T23:16:52.391 DEBUG:teuthology.parallel:result is None 2024-05-08T23:16:52.391 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T23:16:52.400 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T23:16:52.400 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:16:52.402 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T23:16:52.424 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T23:16:52.450 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T23:16:52.477 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T23:16:52.477 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:16:52.480 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T23:16:52.516 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T23:16:52.527 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T23:16:52.528 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T23:16:52.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:16:52.571 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:16:52.573 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T23:16:52.607 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:16:52.659 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:16:52.663 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:16:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T23:16:52.692 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T23:16:52.701 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T23:16:52.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:16:52.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T23:16:52.739 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T23:16:52.752 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T23:16:52.752 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:16:52.779 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T23:16:52.801 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:16:52.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:16:52.948 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:16:53.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:16:53.046 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:16:53.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:16:53.120 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:16:53.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T23:16:53.225 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:16:53.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T23:16:53.364 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:16:53.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T23:16:53.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-05-08T23:16:53.442 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-05-08T23:16:53.513 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:16:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:16:53.950 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T23:16:53.964 INFO:teuthology.task.internal:Starting timer... 2024-05-08T23:16:53.965 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T23:16:53.992 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T23:16:54.016 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T23:16:54.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-05-08T23:16:54.098 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-05-08T23:16:54.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-05-08T23:16:54.172 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-05-08T23:16:54.174 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T23:16:54.174 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-05-08T23:16:54.199 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-05-08T23:16:54.199 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-05-08T23:16:54.221 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-05-08T23:16:54.222 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi042.front.sepia.ceph.com': 'permissive'} 2024-05-08T23:16:54.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:16:54.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:16:54.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:16:54.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:16:54.306 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T23:16:54.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:16:54.342 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-05-08T23:16:54.382 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T23:16:54.396 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T23:16:54.401 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T23:16:54.536 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T23:16:54.552 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T23:16:54.553 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi042.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:28:06.336 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi042.front.sepia.ceph.com')] 2024-05-08T23:28:06.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-08T23:28:06.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:28:06.443 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-08T23:28:06.519 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-08T23:28:06.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-08T23:28:06.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:28:06.601 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-05-08T23:28:06.675 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-05-08T23:28:06.676 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:28:06.686 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:28:06.686 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:28:06.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:28:06.689 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:28:06.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:28:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:28:06.744 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:28:06.756 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:28:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:28:06.835 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-05-08T23:28:06.850 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-05-08T23:28:06.860 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-05-08T23:28:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:28:06.874 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-05-08T23:28:06.885 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:28:06.892 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:28:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:28:06.969 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-08T23:28:06.972 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:28:06.973 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-08T23:28:06.973 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.973 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.973 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.973 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.984 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2024-05-08T23:28:06.991 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:28:06.991 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-05-08T23:28:06.991 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.991 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.992 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.992 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:28:06.992 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:28:07.000 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:28:07.001 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T23:28:07.001 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T23:28:07.001 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:28:07.006 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:28:07.007 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:28:07.007 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:28:07.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:28:07.009 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:28:07.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:28:07.154 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:28:07.155 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:28:07.236 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:28:07.236 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:28:07.267 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:28:07.268 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:28:07.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:28:07.304 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:28:07.304 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:28:07.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:28:07.402 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:28:07.402 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:28:07.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:28:07.439 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:28:07.439 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:28:07.469 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:28:07.501 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-05-08T23:28:07.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-05-08T23:28:07.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:28:07.604 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:28:07.635 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-05-08T23:28:07.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-05-08T23:28:07.937 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2024-05-08T23:28:08.073 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T23:28:09.334 INFO:teuthology.orchestra.run.smithi042.stdout:63 files removed 2024-05-08T23:28:09.359 INFO:teuthology.orchestra.run.smithi033.stdout:63 files removed 2024-05-08T23:28:09.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:28:09.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:28:09.772 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2024-05-08T23:28:09.800 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T23:28:11.705 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 188 kB/s | 80 kB 00:00 2024-05-08T23:28:12.048 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-05-08T23:28:12.064 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 234 kB/s | 80 kB 00:00 2024-05-08T23:28:12.315 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2024-05-08T23:28:12.357 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-05-08T23:28:12.591 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-05-08T23:28:12.639 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2024-05-08T23:28:12.800 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-05-08T23:28:13.447 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 27 MB/s | 16 MB 00:00 2024-05-08T23:28:13.663 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2024-05-08T23:28:17.442 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 291 kB/s | 24 kB 00:00 2024-05-08T23:28:17.666 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 419 kB/s | 24 kB 00:00 2024-05-08T23:28:18.194 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T23:28:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 61 MB 00:01 2024-05-08T23:28:21.303 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 61 MB 00:01 2024-05-08T23:28:31.194 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T23:28:33.995 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-08T23:28:34.462 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 67 MB 00:01 2024-05-08T23:28:50.713 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:28:50.714 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:28:50.734 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:28:50.736 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:28:50.737 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:28:50.739 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:28:50.742 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:28:50.747 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:28:50.748 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:28:50.993 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T23:28:51.009 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:28:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:28:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:28:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:28:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:28:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:28:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:28:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:28:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:28:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:28:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:28:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling module streams: 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql 10 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:Install 130 Packages 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 16 Packages 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:28:51.021 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 215 M 2024-05-08T23:28:51.022 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-08T23:28:51.139 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:28:51.141 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:28:51.162 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:28:51.163 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:28:51.165 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:28:51.166 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:28:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:28:51.174 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:28:51.175 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:28:51.344 INFO:teuthology.orchestra.run.smithi033.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-05-08T23:28:51.429 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout:=================================================================================================================================== 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout:=================================================================================================================================== 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:28:51.445 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:28:51.446 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:28:51.447 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:28:51.448 INFO:teuthology.orchestra.run.smithi042.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:28:51.449 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:28:51.450 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:28:51.451 INFO:teuthology.orchestra.run.smithi042.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:28:51.452 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:28:51.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:28:51.454 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:28:51.455 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:28:51.456 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling module streams: 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql 10 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:=================================================================================================================================== 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:Install 130 Packages 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 16 Packages 2024-05-08T23:28:51.457 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:28:51.458 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 215 M 2024-05-08T23:28:51.458 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-05-08T23:28:51.737 INFO:teuthology.orchestra.run.smithi042.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2024-05-08T23:28:51.869 INFO:teuthology.orchestra.run.smithi033.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.6 MB/s | 860 kB 00:00 2024-05-08T23:28:52.020 INFO:teuthology.orchestra.run.smithi033.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.1 MB/s | 166 kB 00:00 2024-05-08T23:28:52.362 INFO:teuthology.orchestra.run.smithi033.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 4.2 MB/s | 5.5 MB 00:01 2024-05-08T23:28:52.653 INFO:teuthology.orchestra.run.smithi033.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 5.4 MB/s | 1.6 MB 00:00 2024-05-08T23:28:52.696 INFO:teuthology.orchestra.run.smithi042.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 897 kB/s | 860 kB 00:00 2024-05-08T23:28:52.929 INFO:teuthology.orchestra.run.smithi042.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 712 kB/s | 166 kB 00:00 2024-05-08T23:28:53.079 INFO:teuthology.orchestra.run.smithi033.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 2.3 MB 00:01 2024-05-08T23:28:53.354 INFO:teuthology.orchestra.run.smithi033.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 6.9 MB/s | 4.8 MB 00:00 2024-05-08T23:28:55.155 INFO:teuthology.orchestra.run.smithi042.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.0 MB/s | 2.3 MB 00:02 2024-05-08T23:28:55.521 INFO:teuthology.orchestra.run.smithi033.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 6.0 MB/s | 13 MB 00:02 2024-05-08T23:28:55.614 INFO:teuthology.orchestra.run.smithi033.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 267 kB/s | 24 kB 00:00 2024-05-08T23:28:56.264 INFO:teuthology.orchestra.run.smithi042.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 1.4 MB/s | 1.6 MB 00:01 2024-05-08T23:28:59.430 INFO:teuthology.orchestra.run.smithi042.stdout:(6/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 712 kB/s | 5.5 MB 00:07 2024-05-08T23:28:59.497 INFO:teuthology.orchestra.run.smithi042.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 1.5 MB/s | 4.8 MB 00:03 2024-05-08T23:29:00.371 INFO:teuthology.orchestra.run.smithi033.stdout:(10/146): ceph-common-18.2.2-1322.gd21e4f96.el8 2.3 MB/s | 22 MB 00:09 2024-05-08T23:29:00.554 INFO:teuthology.orchestra.run.smithi033.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 261 kB 00:00 2024-05-08T23:29:00.655 INFO:teuthology.orchestra.run.smithi033.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 310 kB/s | 31 kB 00:00 2024-05-08T23:29:00.963 INFO:teuthology.orchestra.run.smithi033.stdout:(13/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 2.3 MB/s | 18 MB 00:07 2024-05-08T23:29:01.005 INFO:teuthology.orchestra.run.smithi033.stdout:(14/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.2 MB/s | 782 kB 00:00 2024-05-08T23:29:01.039 INFO:teuthology.orchestra.run.smithi033.stdout:(15/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.9 MB/s | 142 kB 00:00 2024-05-08T23:29:01.097 INFO:teuthology.orchestra.run.smithi033.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 1.5 MB/s | 139 kB 00:00 2024-05-08T23:29:01.239 INFO:teuthology.orchestra.run.smithi033.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.3 MB/s | 474 kB 00:00 2024-05-08T23:29:01.314 INFO:teuthology.orchestra.run.smithi033.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 597 kB/s | 45 kB 00:00 2024-05-08T23:29:01.406 INFO:teuthology.orchestra.run.smithi033.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.4 MB/s | 129 kB 00:00 2024-05-08T23:29:01.591 INFO:teuthology.orchestra.run.smithi033.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 1.0 MB/s | 197 kB 00:00 2024-05-08T23:29:01.765 INFO:teuthology.orchestra.run.smithi033.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.1 MB/s | 370 kB 00:00 2024-05-08T23:29:01.940 INFO:teuthology.orchestra.run.smithi033.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.0 MB/s | 365 kB 00:00 2024-05-08T23:29:02.041 INFO:teuthology.orchestra.run.smithi033.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.0 MB/s | 107 kB 00:00 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi033.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 954 kB/s | 95 kB 00:00 2024-05-08T23:29:03.433 INFO:teuthology.orchestra.run.smithi033.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.7 MB/s | 3.5 MB 00:01 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi033.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.9 MB/s | 181 kB 00:00 2024-05-08T23:29:03.617 INFO:teuthology.orchestra.run.smithi033.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 265 kB/s | 24 kB 00:00 2024-05-08T23:29:03.858 INFO:teuthology.orchestra.run.smithi033.stdout:(28/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.4 MB/s | 6.5 MB 00:02 2024-05-08T23:29:04.109 INFO:teuthology.orchestra.run.smithi033.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 281 kB/s | 138 kB 00:00 2024-05-08T23:29:04.172 INFO:teuthology.orchestra.run.smithi042.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 2.8 MB/s | 13 MB 00:04 2024-05-08T23:29:04.255 INFO:teuthology.orchestra.run.smithi042.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 292 kB/s | 24 kB 00:00 2024-05-08T23:29:05.826 INFO:teuthology.orchestra.run.smithi033.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.8 MB/s | 3.5 MB 00:01 2024-05-08T23:29:06.026 INFO:teuthology.orchestra.run.smithi033.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.2 MB/s | 245 kB 00:00 2024-05-08T23:29:06.134 INFO:teuthology.orchestra.run.smithi033.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 454 kB/s | 49 kB 00:00 2024-05-08T23:29:06.243 INFO:teuthology.orchestra.run.smithi033.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 151 kB/s | 16 kB 00:00 2024-05-08T23:29:06.452 INFO:teuthology.orchestra.run.smithi033.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.2 MB/s | 263 kB 00:00 2024-05-08T23:29:06.644 INFO:teuthology.orchestra.run.smithi033.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 886 kB/s | 170 kB 00:00 2024-05-08T23:29:06.752 INFO:teuthology.orchestra.run.smithi033.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 167 kB/s | 18 kB 00:00 2024-05-08T23:29:06.836 INFO:teuthology.orchestra.run.smithi033.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 3.6 MB/s | 306 kB 00:00 2024-05-08T23:29:06.870 INFO:teuthology.orchestra.run.smithi033.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2024-05-08T23:29:06.903 INFO:teuthology.orchestra.run.smithi033.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2024-05-08T23:29:06.995 INFO:teuthology.orchestra.run.smithi033.stdout:(40/146): ceph-mgr-diskprediction-local-18.2.2- 2.6 MB/s | 7.4 MB 00:02 2024-05-08T23:29:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T23:29:07.054 INFO:teuthology.orchestra.run.smithi033.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 5.4 MB/s | 185 kB 00:00 2024-05-08T23:29:07.079 INFO:teuthology.orchestra.run.smithi033.stdout:(43/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 520 kB/s | 44 kB 00:00 2024-05-08T23:29:07.105 INFO:teuthology.orchestra.run.smithi033.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 658 kB/s | 33 kB 00:00 2024-05-08T23:29:07.130 INFO:teuthology.orchestra.run.smithi033.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-05-08T23:29:07.155 INFO:teuthology.orchestra.run.smithi033.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2024-05-08T23:29:07.606 INFO:teuthology.orchestra.run.smithi033.stdout:(47/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 3.9 MB/s | 47 MB 00:11 2024-05-08T23:29:07.631 INFO:teuthology.orchestra.run.smithi033.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 171 kB/s | 81 kB 00:00 2024-05-08T23:29:07.656 INFO:teuthology.orchestra.run.smithi033.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 732 kB/s | 384 kB 00:00 2024-05-08T23:29:07.673 INFO:teuthology.orchestra.run.smithi033.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-05-08T23:29:07.690 INFO:teuthology.orchestra.run.smithi033.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2024-05-08T23:29:07.707 INFO:teuthology.orchestra.run.smithi033.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2024-05-08T23:29:07.732 INFO:teuthology.orchestra.run.smithi033.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2024-05-08T23:29:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2024-05-08T23:29:07.783 INFO:teuthology.orchestra.run.smithi033.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 9.2 MB/s | 1.2 MB 00:00 2024-05-08T23:29:07.800 INFO:teuthology.orchestra.run.smithi033.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2024-05-08T23:29:07.816 INFO:teuthology.orchestra.run.smithi033.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2024-05-08T23:29:07.842 INFO:teuthology.orchestra.run.smithi033.stdout:(58/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2024-05-08T23:29:07.876 INFO:teuthology.orchestra.run.smithi033.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2024-05-08T23:29:07.901 INFO:teuthology.orchestra.run.smithi033.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 766 kB/s | 64 kB 00:00 2024-05-08T23:29:07.917 INFO:teuthology.orchestra.run.smithi033.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-05-08T23:29:07.934 INFO:teuthology.orchestra.run.smithi033.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 407 kB/s | 24 kB 00:00 2024-05-08T23:29:07.960 INFO:teuthology.orchestra.run.smithi033.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2024-05-08T23:29:07.985 INFO:teuthology.orchestra.run.smithi033.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 409 kB/s | 27 kB 00:00 2024-05-08T23:29:08.011 INFO:teuthology.orchestra.run.smithi033.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-05-08T23:29:08.036 INFO:teuthology.orchestra.run.smithi033.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2024-05-08T23:29:08.061 INFO:teuthology.orchestra.run.smithi033.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 810 kB/s | 61 kB 00:00 2024-05-08T23:29:08.086 INFO:teuthology.orchestra.run.smithi033.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2024-05-08T23:29:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 924 kB/s | 85 kB 00:00 2024-05-08T23:29:08.129 INFO:teuthology.orchestra.run.smithi033.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-05-08T23:29:08.187 INFO:teuthology.orchestra.run.smithi033.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2024-05-08T23:29:08.429 INFO:teuthology.orchestra.run.smithi033.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 189 kB/s | 44 kB 00:00 2024-05-08T23:29:08.496 INFO:teuthology.orchestra.run.smithi033.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 56 kB/s | 20 kB 00:00 2024-05-08T23:29:08.521 INFO:teuthology.orchestra.run.smithi033.stdout:(74/146): python3-packaging-16.8-9.el8.noarch.r 115 kB/s | 48 kB 00:00 2024-05-08T23:29:08.563 INFO:teuthology.orchestra.run.smithi033.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 823 kB/s | 110 kB 00:00 2024-05-08T23:29:08.664 INFO:teuthology.orchestra.run.smithi033.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 892 kB/s | 126 kB 00:00 2024-05-08T23:29:08.697 INFO:teuthology.orchestra.run.smithi033.stdout:(77/146): python3-jsonpatch-1.21-2.el8.noarch.r 132 kB/s | 27 kB 00:00 2024-05-08T23:29:08.739 INFO:teuthology.orchestra.run.smithi033.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.5 MB/s | 457 kB 00:00 2024-05-08T23:29:08.773 INFO:teuthology.orchestra.run.smithi033.stdout:(79/146): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2024-05-08T23:29:08.823 INFO:teuthology.orchestra.run.smithi033.stdout:(80/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-05-08T23:29:08.890 INFO:teuthology.orchestra.run.smithi033.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2024-05-08T23:29:08.915 INFO:teuthology.orchestra.run.smithi033.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 273 kB/s | 39 kB 00:00 2024-05-08T23:29:08.974 INFO:teuthology.orchestra.run.smithi033.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-05-08T23:29:09.058 INFO:teuthology.orchestra.run.smithi033.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2024-05-08T23:29:09.083 INFO:teuthology.orchestra.run.smithi033.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 619 kB/s | 103 kB 00:00 2024-05-08T23:29:09.217 INFO:teuthology.orchestra.run.smithi033.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-05-08T23:29:09.250 INFO:teuthology.orchestra.run.smithi033.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.7 MB/s | 538 kB 00:00 2024-05-08T23:29:09.400 INFO:teuthology.orchestra.run.smithi033.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-08T23:29:09.417 INFO:teuthology.orchestra.run.smithi033.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-05-08T23:29:09.465 INFO:teuthology.orchestra.run.smithi033.stdout:(90/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-05-08T23:29:09.592 INFO:teuthology.orchestra.run.smithi033.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-05-08T23:29:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:(92/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2024-05-08T23:29:09.743 INFO:teuthology.orchestra.run.smithi033.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 16 MB/s | 5.2 MB 00:00 2024-05-08T23:29:09.785 INFO:teuthology.orchestra.run.smithi033.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 4.4 MB/s | 714 kB 00:00 2024-05-08T23:29:09.902 INFO:teuthology.orchestra.run.smithi033.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-08T23:29:09.919 INFO:teuthology.orchestra.run.smithi033.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-05-08T23:29:09.944 INFO:teuthology.orchestra.run.smithi033.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T23:29:10.003 INFO:teuthology.orchestra.run.smithi033.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 541 kB/s | 54 kB 00:00 2024-05-08T23:29:10.103 INFO:teuthology.orchestra.run.smithi033.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-08T23:29:10.122 INFO:teuthology.orchestra.run.smithi042.stdout:(10/146): ceph-common-18.2.2-1322.gd21e4f96.el8 1.2 MB/s | 22 MB 00:18 2024-05-08T23:29:10.191 INFO:teuthology.orchestra.run.smithi033.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 8.2 MB/s | 1.5 MB 00:00 2024-05-08T23:29:10.254 INFO:teuthology.orchestra.run.smithi033.stdout:(101/146): postgresql-contrib-10.23-4.module+el 5.3 MB/s | 812 kB 00:00 2024-05-08T23:29:10.414 INFO:teuthology.orchestra.run.smithi042.stdout:(11/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 1.7 MB/s | 18 MB 00:10 2024-05-08T23:29:10.435 INFO:teuthology.orchestra.run.smithi033.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-05-08T23:29:10.439 INFO:teuthology.orchestra.run.smithi042.stdout:(12/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 825 kB/s | 261 kB 00:00 2024-05-08T23:29:10.463 INFO:teuthology.orchestra.run.smithi033.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 86 kB/s | 18 kB 00:00 2024-05-08T23:29:10.489 INFO:teuthology.orchestra.run.smithi042.stdout:(13/146): libcephfs-devel-18.2.2-1322.gd21e4f96 414 kB/s | 31 kB 00:00 2024-05-08T23:29:10.530 INFO:teuthology.orchestra.run.smithi033.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 450 kB/s | 43 kB 00:00 2024-05-08T23:29:10.615 INFO:teuthology.orchestra.run.smithi042.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.1 MB/s | 142 kB 00:00 2024-05-08T23:29:10.621 INFO:teuthology.orchestra.run.smithi033.stdout:(105/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-05-08T23:29:10.647 INFO:teuthology.orchestra.run.smithi033.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 843 kB/s | 155 kB 00:00 2024-05-08T23:29:10.682 INFO:teuthology.orchestra.run.smithi042.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 2.0 MB/s | 139 kB 00:00 2024-05-08T23:29:10.689 INFO:teuthology.orchestra.run.smithi033.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 687 kB/s | 109 kB 00:00 2024-05-08T23:29:10.731 INFO:teuthology.orchestra.run.smithi033.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 392 kB/s | 43 kB 00:00 2024-05-08T23:29:10.781 INFO:teuthology.orchestra.run.smithi033.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 140 kB/s | 19 kB 00:00 2024-05-08T23:29:10.798 INFO:teuthology.orchestra.run.smithi042.stdout:(16/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 2.1 MB/s | 782 kB 00:00 2024-05-08T23:29:10.807 INFO:teuthology.orchestra.run.smithi033.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 374 kB/s | 44 kB 00:00 2024-05-08T23:29:10.824 INFO:teuthology.orchestra.run.smithi042.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 3.3 MB/s | 474 kB 00:00 2024-05-08T23:29:10.840 INFO:teuthology.orchestra.run.smithi033.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2024-05-08T23:29:10.873 INFO:teuthology.orchestra.run.smithi033.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2024-05-08T23:29:10.899 INFO:teuthology.orchestra.run.smithi042.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 596 kB/s | 45 kB 00:00 2024-05-08T23:29:10.907 INFO:teuthology.orchestra.run.smithi033.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2024-05-08T23:29:10.966 INFO:teuthology.orchestra.run.smithi042.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.9 MB/s | 129 kB 00:00 2024-05-08T23:29:10.974 INFO:teuthology.orchestra.run.smithi033.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2024-05-08T23:29:11.010 INFO:teuthology.orchestra.run.smithi033.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 359 kB/s | 48 kB 00:00 2024-05-08T23:29:11.032 INFO:teuthology.orchestra.run.smithi042.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.9 MB/s | 197 kB 00:00 2024-05-08T23:29:11.058 INFO:teuthology.orchestra.run.smithi033.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 578 kB/s | 87 kB 00:00 2024-05-08T23:29:11.074 INFO:teuthology.orchestra.run.smithi033.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 574 kB/s | 58 kB 00:00 2024-05-08T23:29:11.091 INFO:teuthology.orchestra.run.smithi033.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 460 kB/s | 39 kB 00:00 2024-05-08T23:29:11.175 INFO:teuthology.orchestra.run.smithi042.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.6 MB/s | 370 kB 00:00 2024-05-08T23:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 586 kB/s | 83 kB 00:00 2024-05-08T23:29:11.300 INFO:teuthology.orchestra.run.smithi042.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.9 MB/s | 365 kB 00:00 2024-05-08T23:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.4 MB/s | 645 kB 00:00 2024-05-08T23:29:11.375 INFO:teuthology.orchestra.run.smithi042.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 1.4 MB/s | 107 kB 00:00 2024-05-08T23:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2024-05-08T23:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 529 kB/s | 172 kB 00:00 2024-05-08T23:29:11.459 INFO:teuthology.orchestra.run.smithi042.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 95 kB 00:00 2024-05-08T23:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 414 kB/s | 73 kB 00:00 2024-05-08T23:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 434 kB/s | 62 kB 00:00 2024-05-08T23:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.5 MB/s | 562 kB 00:00 2024-05-08T23:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 15 MB/s | 2.3 MB 00:00 2024-05-08T23:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 296 kB/s | 47 kB 00:00 2024-05-08T23:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.3 MB/s | 559 kB 00:00 2024-05-08T23:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2024-05-08T23:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 479 kB/s | 100 kB 00:00 2024-05-08T23:29:11.943 INFO:teuthology.orchestra.run.smithi042.stdout:(25/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 6.1 MB/s | 47 MB 00:07 2024-05-08T23:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.8 MB/s | 232 kB 00:00 2024-05-08T23:29:12.043 INFO:teuthology.orchestra.run.smithi042.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.8 MB/s | 181 kB 00:00 2024-05-08T23:29:12.110 INFO:teuthology.orchestra.run.smithi042.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 366 kB/s | 24 kB 00:00 2024-05-08T23:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 4.0 MB/s | 303 kB 00:00 2024-05-08T23:29:12.168 INFO:teuthology.orchestra.run.smithi042.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.3 MB/s | 138 kB 00:00 2024-05-08T23:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 8.4 MB/s | 854 kB 00:00 2024-05-08T23:29:12.361 INFO:teuthology.orchestra.run.smithi042.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 3.8 MB/s | 3.5 MB 00:00 2024-05-08T23:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 2.9 MB/s | 711 kB 00:00 2024-05-08T23:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout:(135/146): librados2-18.2.2-1322.gd21e4f96.el8. 4.8 MB/s | 3.7 MB 00:00 2024-05-08T23:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 13 MB/s | 1.5 MB 00:00 2024-05-08T23:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-05-08T23:29:12.731 INFO:teuthology.orchestra.run.smithi033.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-05-08T23:29:12.752 INFO:teuthology.orchestra.run.smithi042.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 6.0 MB/s | 3.5 MB 00:00 2024-05-08T23:29:12.755 INFO:teuthology.orchestra.run.smithi033.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-08T23:29:12.819 INFO:teuthology.orchestra.run.smithi042.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 3.6 MB/s | 245 kB 00:00 2024-05-08T23:29:12.839 INFO:teuthology.orchestra.run.smithi033.stdout:(140/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 2.8 MB/s | 237 kB 00:00 2024-05-08T23:29:12.878 INFO:teuthology.orchestra.run.smithi042.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 840 kB/s | 49 kB 00:00 2024-05-08T23:29:12.898 INFO:teuthology.orchestra.run.smithi033.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 254 kB 00:00 2024-05-08T23:29:12.937 INFO:teuthology.orchestra.run.smithi042.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 279 kB/s | 16 kB 00:00 2024-05-08T23:29:12.956 INFO:teuthology.orchestra.run.smithi033.stdout:(142/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 3.6 MB 00:01 2024-05-08T23:29:12.982 INFO:teuthology.orchestra.run.smithi033.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-08T23:29:12.999 INFO:teuthology.orchestra.run.smithi033.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 985 kB/s | 99 kB 00:00 2024-05-08T23:29:13.004 INFO:teuthology.orchestra.run.smithi042.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.9 MB/s | 263 kB 00:00 2024-05-08T23:29:13.045 INFO:teuthology.orchestra.run.smithi042.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 4.0 MB/s | 170 kB 00:00 2024-05-08T23:29:13.087 INFO:teuthology.orchestra.run.smithi042.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 435 kB/s | 18 kB 00:00 2024-05-08T23:29:13.090 INFO:teuthology.orchestra.run.smithi033.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 948 kB/s | 103 kB 00:00 2024-05-08T23:29:13.149 INFO:teuthology.orchestra.run.smithi033.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 13 MB/s | 2.5 MB 00:00 2024-05-08T23:29:13.152 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:29:13.152 INFO:teuthology.orchestra.run.smithi033.stdout:Total 9.7 MB/s | 215 MB 00:22 2024-05-08T23:29:13.213 INFO:teuthology.orchestra.run.smithi042.stdout:(37/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 6.5 MB 00:02 2024-05-08T23:29:13.288 INFO:teuthology.orchestra.run.smithi042.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2024-05-08T23:29:13.405 INFO:teuthology.orchestra.run.smithi042.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 644 kB/s | 75 kB 00:00 2024-05-08T23:29:13.430 INFO:teuthology.orchestra.run.smithi042.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 271 kB/s | 59 kB 00:00 2024-05-08T23:29:13.489 INFO:teuthology.orchestra.run.smithi042.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2024-05-08T23:29:13.523 INFO:teuthology.orchestra.run.smithi042.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-05-08T23:29:13.648 INFO:teuthology.orchestra.run.smithi042.stdout:(43/146): ceph-mgr-diskprediction-local-18.2.2- 5.8 MB/s | 7.4 MB 00:01 2024-05-08T23:29:13.673 INFO:teuthology.orchestra.run.smithi042.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 219 kB/s | 33 kB 00:00 2024-05-08T23:29:13.699 INFO:teuthology.orchestra.run.smithi042.stdout:(45/146): python3-beautifulsoup4-4.6.3-2.el8.1. 886 kB/s | 185 kB 00:00 2024-05-08T23:29:13.724 INFO:teuthology.orchestra.run.smithi042.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2024-05-08T23:29:13.758 INFO:teuthology.orchestra.run.smithi042.stdout:(47/146): python3-certifi-2018.10.15-7.el8.noar 149 kB/s | 16 kB 00:00 2024-05-08T23:29:13.783 INFO:teuthology.orchestra.run.smithi042.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2024-05-08T23:29:13.808 INFO:teuthology.orchestra.run.smithi042.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.4 MB/s | 384 kB 00:00 2024-05-08T23:29:13.825 INFO:teuthology.orchestra.run.smithi042.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-05-08T23:29:13.842 INFO:teuthology.orchestra.run.smithi042.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 299 kB/s | 18 kB 00:00 2024-05-08T23:29:13.867 INFO:teuthology.orchestra.run.smithi042.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2024-05-08T23:29:13.926 INFO:teuthology.orchestra.run.smithi042.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 9.9 MB/s | 1.2 MB 00:00 2024-05-08T23:29:13.942 INFO:teuthology.orchestra.run.smithi042.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 585 kB/s | 59 kB 00:00 2024-05-08T23:29:14.085 INFO:teuthology.orchestra.run.smithi042.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 278 kB/s | 60 kB 00:00 2024-05-08T23:29:14.160 INFO:teuthology.orchestra.run.smithi042.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 72 kB/s | 16 kB 00:00 2024-05-08T23:29:14.177 INFO:teuthology.orchestra.run.smithi042.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2024-05-08T23:29:14.202 INFO:teuthology.orchestra.run.smithi042.stdout:(58/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2024-05-08T23:29:14.227 INFO:teuthology.orchestra.run.smithi042.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T23:29:14.244 INFO:teuthology.orchestra.run.smithi042.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 448 kB/s | 19 kB 00:00 2024-05-08T23:29:14.270 INFO:teuthology.orchestra.run.smithi042.stdout:(61/146): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-05-08T23:29:14.295 INFO:teuthology.orchestra.run.smithi042.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 355 kB/s | 24 kB 00:00 2024-05-08T23:29:14.311 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-08T23:29:14.346 INFO:teuthology.orchestra.run.smithi042.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 290 kB/s | 29 kB 00:00 2024-05-08T23:29:14.370 INFO:teuthology.orchestra.run.smithi042.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2024-05-08T23:29:14.396 INFO:teuthology.orchestra.run.smithi042.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2024-05-08T23:29:14.413 INFO:teuthology.orchestra.run.smithi042.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2024-05-08T23:29:14.438 INFO:teuthology.orchestra.run.smithi042.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 911 kB/s | 61 kB 00:00 2024-05-08T23:29:14.463 INFO:teuthology.orchestra.run.smithi042.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2024-05-08T23:29:14.488 INFO:teuthology.orchestra.run.smithi042.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 924 kB/s | 85 kB 00:00 2024-05-08T23:29:14.514 INFO:teuthology.orchestra.run.smithi042.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T23:29:14.581 INFO:teuthology.orchestra.run.smithi042.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2024-05-08T23:29:14.606 INFO:teuthology.orchestra.run.smithi042.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 170 kB/s | 20 kB 00:00 2024-05-08T23:29:14.698 INFO:teuthology.orchestra.run.smithi042.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 251 kB/s | 44 kB 00:00 2024-05-08T23:29:14.732 INFO:teuthology.orchestra.run.smithi042.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 731 kB/s | 110 kB 00:00 2024-05-08T23:29:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-08T23:29:14.762 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-08T23:29:14.765 INFO:teuthology.orchestra.run.smithi042.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 177 kB/s | 27 kB 00:00 2024-05-08T23:29:14.857 INFO:teuthology.orchestra.run.smithi042.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 796 kB/s | 126 kB 00:00 2024-05-08T23:29:14.891 INFO:teuthology.orchestra.run.smithi042.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2024-05-08T23:29:14.933 INFO:teuthology.orchestra.run.smithi042.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2024-05-08T23:29:14.966 INFO:teuthology.orchestra.run.smithi042.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2024-05-08T23:29:15.063 INFO:teuthology.orchestra.run.smithi042.stdout:(80/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.6 MB/s | 259 kB 00:00 2024-05-08T23:29:15.092 INFO:teuthology.orchestra.run.smithi042.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 244 kB/s | 39 kB 00:00 2024-05-08T23:29:15.134 INFO:teuthology.orchestra.run.smithi042.stdout:(82/146): libtool-2.4.6-25.el8.x86_64.rpm 2.9 MB/s | 709 kB 00:00 2024-05-08T23:29:15.218 INFO:teuthology.orchestra.run.smithi042.stdout:(83/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 827 kB/s | 103 kB 00:00 2024-05-08T23:29:15.243 INFO:teuthology.orchestra.run.smithi042.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 357 kB/s | 64 kB 00:00 2024-05-08T23:29:15.368 INFO:teuthology.orchestra.run.smithi042.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-05-08T23:29:15.477 INFO:teuthology.orchestra.run.smithi042.stdout:(86/146): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2024-05-08T23:29:15.510 INFO:teuthology.orchestra.run.smithi042.stdout:(87/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-05-08T23:29:15.536 INFO:teuthology.orchestra.run.smithi042.stdout:(88/146): python3-jinja2-2.10.1-3.el8.noarch.rp 1.7 MB/s | 538 kB 00:00 2024-05-08T23:29:15.602 INFO:teuthology.orchestra.run.smithi042.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2024-05-08T23:29:15.686 INFO:teuthology.orchestra.run.smithi042.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2024-05-08T23:29:15.803 INFO:teuthology.orchestra.run.smithi042.stdout:(91/146): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2024-05-08T23:29:15.829 INFO:teuthology.orchestra.run.smithi042.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 695 kB/s | 157 kB 00:00 2024-05-08T23:29:15.962 INFO:teuthology.orchestra.run.smithi042.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-05-08T23:29:15.987 INFO:teuthology.orchestra.run.smithi042.stdout:(94/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-08T23:29:16.021 INFO:teuthology.orchestra.run.smithi042.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 3.2 MB/s | 714 kB 00:00 2024-05-08T23:29:16.083 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-08T23:29:16.095 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-08T23:29:16.105 INFO:teuthology.orchestra.run.smithi042.stdout:(96/146): python3-pytz-2017.2-11.el8.noarch.rpm 464 kB/s | 54 kB 00:00 2024-05-08T23:29:16.180 INFO:teuthology.orchestra.run.smithi042.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 932 kB/s | 202 kB 00:00 2024-05-08T23:29:16.289 INFO:teuthology.orchestra.run.smithi042.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-08T23:29:16.322 INFO:teuthology.orchestra.run.smithi042.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 11 MB/s | 1.5 MB 00:00 2024-05-08T23:29:16.573 INFO:teuthology.orchestra.run.smithi042.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 25 MB/s | 14 MB 00:00 2024-05-08T23:29:16.614 INFO:teuthology.orchestra.run.smithi042.stdout:(101/146): postgresql-contrib-10.23-4.module+el 2.4 MB/s | 812 kB 00:00 2024-05-08T23:29:16.740 INFO:teuthology.orchestra.run.smithi042.stdout:(102/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-05-08T23:29:16.765 INFO:teuthology.orchestra.run.smithi042.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 119 kB/s | 18 kB 00:00 2024-05-08T23:29:16.790 INFO:teuthology.orchestra.run.smithi042.stdout:(104/146): python3-packaging-16.8-9.el8.noarch. 222 kB/s | 48 kB 00:00 2024-05-08T23:29:16.857 INFO:teuthology.orchestra.run.smithi042.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 364 kB/s | 43 kB 00:00 2024-05-08T23:29:16.882 INFO:teuthology.orchestra.run.smithi042.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.3 MB/s | 155 kB 00:00 2024-05-08T23:29:16.941 INFO:teuthology.orchestra.run.smithi042.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 725 kB/s | 109 kB 00:00 2024-05-08T23:29:16.983 INFO:teuthology.orchestra.run.smithi042.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 340 kB/s | 43 kB 00:00 2024-05-08T23:29:17.008 INFO:teuthology.orchestra.run.smithi042.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2024-05-08T23:29:17.092 INFO:teuthology.orchestra.run.smithi042.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 291 kB/s | 44 kB 00:00 2024-05-08T23:29:17.150 INFO:teuthology.orchestra.run.smithi042.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-05-08T23:29:17.184 INFO:teuthology.orchestra.run.smithi042.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.9 MB/s | 605 kB 00:00 2024-05-08T23:29:17.292 INFO:teuthology.orchestra.run.smithi042.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-05-08T23:29:17.318 INFO:teuthology.orchestra.run.smithi042.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-05-08T23:29:17.343 INFO:teuthology.orchestra.run.smithi042.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 303 kB/s | 48 kB 00:00 2024-05-08T23:29:17.377 INFO:teuthology.orchestra.run.smithi042.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-05-08T23:29:17.402 INFO:teuthology.orchestra.run.smithi042.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 690 kB/s | 58 kB 00:00 2024-05-08T23:29:17.469 INFO:teuthology.orchestra.run.smithi042.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 307 kB/s | 39 kB 00:00 2024-05-08T23:29:17.511 INFO:teuthology.orchestra.run.smithi042.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 621 kB/s | 83 kB 00:00 2024-05-08T23:29:17.545 INFO:teuthology.orchestra.run.smithi042.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-05-08T23:29:17.612 INFO:teuthology.orchestra.run.smithi042.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-05-08T23:29:17.687 INFO:teuthology.orchestra.run.smithi042.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-05-08T23:29:17.745 INFO:teuthology.orchestra.run.smithi042.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 462 kB/s | 62 kB 00:00 2024-05-08T23:29:17.771 INFO:teuthology.orchestra.run.smithi042.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 321 kB/s | 73 kB 00:00 2024-05-08T23:29:17.846 INFO:teuthology.orchestra.run.smithi042.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.5 MB/s | 562 kB 00:00 2024-05-08T23:29:17.897 INFO:teuthology.orchestra.run.smithi042.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 375 kB/s | 47 kB 00:00 2024-05-08T23:29:17.971 INFO:teuthology.orchestra.run.smithi042.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 10 MB/s | 2.3 MB 00:00 2024-05-08T23:29:18.030 INFO:teuthology.orchestra.run.smithi042.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.0 MB/s | 559 kB 00:00 2024-05-08T23:29:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:(129/146): NetworkManager-initscripts-updown-1. 910 kB/s | 144 kB 00:00 2024-05-08T23:29:18.106 INFO:teuthology.orchestra.run.smithi042.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 745 kB/s | 100 kB 00:00 2024-05-08T23:29:18.223 INFO:teuthology.orchestra.run.smithi042.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.9 MB/s | 232 kB 00:00 2024-05-08T23:29:18.390 INFO:teuthology.orchestra.run.smithi042.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-05-08T23:29:18.745 INFO:teuthology.orchestra.run.smithi042.stdout:(133/146): librados2-18.2.2-1322.gd21e4f96.el8. 5.2 MB/s | 3.7 MB 00:00 2024-05-08T23:29:18.815 INFO:teuthology.orchestra.run.smithi042.stdout:(134/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 4.7 MB/s | 3.6 MB 00:00 2024-05-08T23:29:18.849 INFO:teuthology.orchestra.run.smithi042.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 1.8 MB/s | 854 kB 00:00 2024-05-08T23:29:18.933 INFO:teuthology.orchestra.run.smithi042.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.7 MB/s | 711 kB 00:00 2024-05-08T23:29:18.991 INFO:teuthology.orchestra.run.smithi042.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-08T23:29:19.025 INFO:teuthology.orchestra.run.smithi042.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-05-08T23:29:19.100 INFO:teuthology.orchestra.run.smithi042.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.4 MB/s | 1.1 MB 00:00 2024-05-08T23:29:19.125 INFO:teuthology.orchestra.run.smithi042.stdout:(140/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 221 kB 00:00 2024-05-08T23:29:19.167 INFO:teuthology.orchestra.run.smithi042.stdout:(141/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T23:29:19.209 INFO:teuthology.orchestra.run.smithi042.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 2.1 MB/s | 180 kB 00:00 2024-05-08T23:29:19.235 INFO:teuthology.orchestra.run.smithi042.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-08T23:29:19.302 INFO:teuthology.orchestra.run.smithi042.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 739 kB/s | 99 kB 00:00 2024-05-08T23:29:19.343 INFO:teuthology.orchestra.run.smithi042.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 947 kB/s | 103 kB 00:00 2024-05-08T23:29:19.402 INFO:teuthology.orchestra.run.smithi042.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 13 MB/s | 2.5 MB 00:00 2024-05-08T23:29:19.405 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:29:19.405 INFO:teuthology.orchestra.run.smithi042.stdout:Total 7.7 MB/s | 215 MB 00:27 2024-05-08T23:29:20.569 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-05-08T23:29:21.019 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-05-08T23:29:21.019 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-05-08T23:29:21.640 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-08T23:29:21.788 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:29:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:29:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:29:22.338 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-05-08T23:29:22.349 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-05-08T23:29:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:29:22.626 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:29:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:29:22.867 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:29:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:29:23.145 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:29:23.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:29:23.371 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:29:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:29:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:29:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:29:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:29:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:29:25.009 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:29:25.144 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:29:25.327 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:29:25.428 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:29:25.720 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:29:25.961 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:29:26.043 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:29:26.221 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:29:26.475 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:29:26.632 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:29:27.069 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:29:27.298 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:29:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:29:27.536 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:29:27.700 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:29:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:29:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:29:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:29:28.215 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-05-08T23:29:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:29:28.364 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:29:28.789 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:29:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:29:29.026 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:29:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:29:29.274 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:29:29.468 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:29:29.547 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:29:29.702 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:29:29.736 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:29:29.757 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:29:29.916 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:29:29.979 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:29:30.127 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:29:30.216 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:29:30.409 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:29:30.643 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:29:30.690 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:29:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:29:31.083 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:29:31.092 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:29:31.248 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:29:31.293 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:29:31.549 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:29:31.721 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:29:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:29:31.885 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:29:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:29:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:29:32.148 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:29:32.230 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:29:32.232 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:29:32.372 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:29:32.520 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:29:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:29:32.776 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:29:32.854 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:29:33.028 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:29:33.257 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:29:33.465 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:29:33.702 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:29:33.855 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:29:34.042 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:29:34.048 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:29:34.193 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:29:34.289 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:29:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:29:34.668 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:29:34.943 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:29:35.088 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:29:35.226 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:29:35.943 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:29:36.126 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:29:36.514 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:29:36.789 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:29:36.815 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:29:36.951 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:29:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:29:37.214 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:29:37.467 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:29:37.642 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:29:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:29:37.832 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:29:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:29:37.983 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:29:38.123 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:29:38.160 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:29:38.199 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:29:38.316 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:29:38.450 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:29:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:29:38.686 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:29:38.882 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:29:38.961 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:29:39.204 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:29:39.358 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:29:39.364 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:29:39.498 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:29:39.666 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:29:39.686 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:29:39.765 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:29:39.870 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T23:29:40.067 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T23:29:40.272 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T23:29:40.445 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T23:29:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T23:29:40.847 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:29:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T23:29:41.220 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:29:41.245 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T23:29:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T23:29:41.534 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:29:41.643 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T23:29:41.724 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:29:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T23:29:42.262 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T23:29:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T23:29:42.862 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T23:29:43.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T23:29:43.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T23:29:43.524 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T23:29:43.693 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:29:43.758 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T23:29:43.976 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T23:29:44.034 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:29:44.201 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T23:29:44.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:29:44.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T23:29:44.424 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:29:44.604 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T23:29:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:29:44.798 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:29:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:29:44.883 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T23:29:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:29:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:29:45.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T23:29:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:29:45.400 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T23:29:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:29:45.589 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:29:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:29:45.652 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T23:29:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:29:45.856 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:29:46.040 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T23:29:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:29:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:29:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:29:46.574 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:29:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:29:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:29:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:29:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:29:47.060 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:29:47.122 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T23:29:47.264 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:29:47.327 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T23:29:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:29:47.540 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T23:29:47.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:29:47.721 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T23:29:47.835 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:29:47.939 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T23:29:48.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:29:48.235 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T23:29:48.381 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:29:48.489 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T23:29:48.542 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:29:48.695 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:29:48.722 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T23:29:48.973 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:29:49.001 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T23:29:49.169 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:29:49.219 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T23:29:49.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:29:49.626 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:29:49.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T23:29:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:29:49.949 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T23:29:50.056 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:29:50.251 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:29:50.280 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T23:29:50.493 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T23:29:50.674 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T23:29:50.941 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T23:29:51.335 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T23:29:51.431 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:29:51.569 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T23:29:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T23:29:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:29:51.973 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T23:29:52.001 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:29:52.172 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T23:29:52.346 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:29:52.397 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-08T23:29:52.574 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:29:52.574 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:29:52.682 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:29:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:29:52.952 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T23:29:53.126 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T23:29:53.333 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:29:53.409 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:29:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T23:29:53.598 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:29:53.716 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T23:29:53.754 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:29:53.833 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:29:53.852 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T23:29:53.976 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:29:54.030 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:29:54.183 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:29:54.413 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:29:54.607 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:29:54.820 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:29:54.978 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:29:55.212 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:29:55.447 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:29:55.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:29:55.844 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:29:55.980 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:29:56.232 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:29:56.429 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:29:56.639 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:29:56.879 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:29:57.108 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:29:57.308 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:29:57.536 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:29:58.156 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:29:58.371 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:29:58.721 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:29:59.009 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:29:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:29:59.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:29:59.309 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:29:59.423 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:29:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:29:59.589 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:29:59.859 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:30:00.036 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:30:00.850 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:30:00.998 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:30:01.062 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:30:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:30:05.397 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:30:06.158 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:30:06.818 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:30:07.419 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:30:07.512 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:30:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:30:21.876 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /run 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-05-08T23:30:21.877 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-05-08T23:30:21.878 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-08T23:30:21.878 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:30:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:30:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:30:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:30:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:30:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:30:25.201 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:30:25.366 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:30:25.389 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:30:27.600 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:30:27.750 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:30:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:30:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:30:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:30:29.524 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:30:29.763 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /run 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2024-05-08T23:30:30.128 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:30:30.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:30:30.846 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:30:31.023 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:30:31.134 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:30:31.533 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:30:31.696 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:30:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:30:31.907 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:30:31.921 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:30:32.323 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:30:32.515 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:30:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:30:32.926 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:30:33.313 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:30:33.453 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:30:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:30:33.532 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:30:33.586 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:30:33.607 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:30:33.698 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:30:33.751 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:30:33.890 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:30:34.276 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:30:34.423 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:30:35.745 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:30:35.943 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:30:36.069 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:30:36.315 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:30:36.566 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:30:37.706 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:30:37.939 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:30:39.010 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:30:39.185 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:30:39.294 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:30:39.376 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:30:39.842 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:30:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:30:39.989 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:30:40.035 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:30:40.208 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:30:40.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:30:40.226 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:30:40.397 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:30:41.215 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:30:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:30:41.591 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:30:41.747 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:30:41.787 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:30:41.805 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:30:41.968 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:30:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:30:42.019 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:30:42.143 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:30:42.201 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:30:42.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:30:42.530 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:30:42.573 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:30:42.685 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:30:45.134 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:45.405 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:30:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:30:46.434 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:30:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:30:46.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:30:47.127 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:30:47.356 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:30:47.546 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:30:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:30:47.610 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:30:47.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:30:47.987 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:30:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:30:48.074 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:30:48.287 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:30:48.303 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:30:48.381 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:30:48.479 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:30:48.570 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:30:48.648 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:30:48.800 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:30:48.800 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:30:49.082 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:30:49.082 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:30:49.291 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:30:49.291 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:30:49.393 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:30:49.578 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:30:49.731 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:30:49.789 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:30:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:30:50.009 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:30:50.189 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:30:50.189 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:30:50.217 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:30:50.398 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:30:50.456 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:30:50.456 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:30:50.615 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:30:50.698 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:30:50.698 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:30:50.802 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:30:50.948 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:30:50.948 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:30:51.181 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:30:51.181 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:30:51.423 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:30:51.565 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:30:51.565 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:30:52.668 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:30:52.669 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:30:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:30:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:30:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:30:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:30:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:30:52.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:30:52.684 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:30:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T23:30:52.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T23:30:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T23:30:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T23:30:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T23:30:52.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:30:52.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:30:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:30:52.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:30:53.446 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:53.767 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:53.808 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:30:54.257 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:30:54.620 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:30:54.889 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:30:55.060 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:30:55.268 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:30:55.496 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:30:55.760 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:30:55.949 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:30:55.961 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:30:56.349 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:30:56.437 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:30:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:30:56.791 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:30:56.942 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:30:57.135 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:30:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:30:57.262 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:30:57.553 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:30:57.553 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:30:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:30:57.770 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:30:57.982 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:30:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:30:58.353 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:30:58.585 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:30:58.585 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:30:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:30:58.855 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:30:59.077 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:30:59.077 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:30:59.310 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:30:59.311 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:30:59.526 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:30:59.527 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:30:59.760 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:31:00.094 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:31:00.094 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:31:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:31:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:31:01.101 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:31:01.102 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:31:01.103 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:31:01.104 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:31:01.105 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:31:01.106 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T23:31:01.107 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T23:31:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T23:31:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T23:31:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T23:31:01.108 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T23:31:01.109 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T23:31:01.110 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T23:31:01.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:31:01.112 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:31:01.113 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:31:01.114 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:31:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:31:06.972 INFO:teuthology.orchestra.run.smithi033.stdout:Installed products updated. 2024-05-08T23:31:07.660 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.661 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:31:07.664 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:31:07.665 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:31:07.666 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:31:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:31:07.668 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:31:07.669 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:31:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:31:07.671 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:31:07.672 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:31:07.672 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:07.672 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T23:31:07.940 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:12.740 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:31:12.740 INFO:teuthology.orchestra.run.smithi042.stdout:Installed products updated. 2024-05-08T23:31:14.325 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:14.325 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.326 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.327 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.328 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:31:14.329 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:31:14.330 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:31:14.331 INFO:teuthology.orchestra.run.smithi042.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:31:14.332 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:31:14.333 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:31:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.335 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:31:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:14.338 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-05-08T23:31:15.094 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:15.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:31:15.321 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:31:15.352 INFO:teuthology.orchestra.run.smithi033.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:31:15.353 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:31:15.353 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:31:15.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:31:15.505 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:31:15.535 INFO:teuthology.orchestra.run.smithi042.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:31:15.535 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:31:15.535 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:31:15.537 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:31:15.537 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:15.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:31:15.591 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:15.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:31:15.644 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:31:15.645 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:15.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:31:15.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:31:15.755 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:15.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:31:15.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:31:15.865 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:31:15.866 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:15.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:31:15.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:31:15.974 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:15.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:31:16.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:31:16.085 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:31:16.085 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:16.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:31:16.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:31:16.193 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:16.194 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:31:16.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:31:16.303 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T23:31:16.385 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T23:31:16.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-05-08T23:31:16.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-05-08T23:31:16.422 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T23:31:16.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:31:16.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T23:31:16.495 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T23:31:16.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:31:16.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T23:31:16.592 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T23:31:16.593 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T23:31:16.593 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T23:31:16.593 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:16.593 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:31:16.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:16.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:16.788 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:16.788 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T23:31:16.810 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:16.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:16.974 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T23:31:16.974 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T23:31:16.975 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175', 'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7698175', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240508.150423-reef'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240508.150423-reef', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2738, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'sleep_before_teardown': 0, 'subset': '17/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240508.150423-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKu9zWuOI0lYhRv2seQduy2OCDHF5sybPrgIHTaBs9ah3XxHHZrIWwwtTYbii8CQZeuNlPU5zhms4XBe3hKyXDY=', 'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFny0qGZEQajeVqgn29XdrGECbC5fjbeuf9BUOXV7JZuevDcuFXUbrzfTvH2dPI1Y5cOfVojETcNqNX/otdtiOo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_19:22:21', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T23:31:16.975 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:31:16.978 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T23:31:17.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:31:17.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T23:31:17.104 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:17.104 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:31:17.133 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:31:17.133 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-05-08T23:31:17.195 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:31:17.195 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:17.195 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 183343 Links: 1 2024-05-08T23:31:17.195 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:17.195 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:17.196 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 23:30:52.258356700 +0000 2024-05-08T23:31:17.196 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 23:27:01.367274549 +0000 2024-05-08T23:31:17.196 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 23:27:01.367274549 +0000 2024-05-08T23:31:17.196 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T23:31:17.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:31:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T23:31:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T23:31:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000236907 s, 2.2 MB/s 2024-05-08T23:31:17.270 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:31:17.333 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-05-08T23:31:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:31:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:17.395 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 180822 Links: 1 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 23:30:52.258356700 +0000 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 23:27:01.738268621 +0000 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 23:27:01.738268621 +0000 2024-05-08T23:31:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T23:31:17.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:31:17.468 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T23:31:17.468 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T23:31:17.468 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00023768 s, 2.2 MB/s 2024-05-08T23:31:17.469 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:31:17.532 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 178060 Links: 1 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:17.594 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 23:30:52.258356700 +0000 2024-05-08T23:31:17.595 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 23:27:02.104262773 +0000 2024-05-08T23:31:17.595 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 23:27:02.104262773 +0000 2024-05-08T23:31:17.595 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T23:31:17.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:31:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T23:31:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T23:31:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000224563 s, 2.3 MB/s 2024-05-08T23:31:17.669 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:31:17.732 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-05-08T23:31:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:31:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 184484 Links: 1 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 23:30:52.258356700 +0000 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 23:27:02.480256764 +0000 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 23:27:02.480256764 +0000 2024-05-08T23:31:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T23:31:17.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:31:17.867 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T23:31:17.867 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T23:31:17.867 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000218719 s, 2.3 MB/s 2024-05-08T23:31:17.869 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:31:17.932 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:17.932 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:17.932 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:31:17.954 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:31:17.954 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-05-08T23:31:18.016 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:31:18.016 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 177886 Links: 1 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-08 23:31:00.668006134 +0000 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-08 23:27:14.599275484 +0000 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-08 23:27:14.599275484 +0000 2024-05-08T23:31:18.017 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-05-08T23:31:18.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:31:18.089 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-08T23:31:18.089 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-08T23:31:18.089 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000132795 s, 3.9 MB/s 2024-05-08T23:31:18.091 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:31:18.154 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-05-08T23:31:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:31:18.217 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:18.218 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 177024 Links: 1 2024-05-08T23:31:18.218 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:18.218 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:18.218 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-08 23:31:00.669006126 +0000 2024-05-08T23:31:18.218 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-08 23:27:14.955272694 +0000 2024-05-08T23:31:18.219 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-08 23:27:14.955272694 +0000 2024-05-08T23:31:18.219 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-05-08T23:31:18.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:31:18.290 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-08T23:31:18.290 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-08T23:31:18.290 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000125961 s, 4.1 MB/s 2024-05-08T23:31:18.292 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:31:18.355 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 181485 Links: 1 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-08 23:31:00.669006126 +0000 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-08 23:27:15.329269760 +0000 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-08 23:27:15.329269760 +0000 2024-05-08T23:31:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-05-08T23:31:18.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:31:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-08T23:31:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-08T23:31:18.491 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000118785 s, 4.3 MB/s 2024-05-08T23:31:18.492 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:31:18.555 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-05-08T23:31:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 183351 Links: 1 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-05-08 23:31:00.669006126 +0000 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-05-08 23:27:15.707266791 +0000 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-05-08 23:27:15.707266791 +0000 2024-05-08T23:31:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-05-08T23:31:18.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:31:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-05-08T23:31:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-05-08T23:31:18.690 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000127722 s, 4.0 MB/s 2024-05-08T23:31:18.691 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:31:18.754 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:18.754 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:31:18.754 INFO:tasks.ceph:Generating config... 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T23:31:18.756 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T23:31:18.756 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-05-08T23:31:18.756 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-05-08T23:31:18.757 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T23:31:18.758 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T23:31:18.759 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T23:31:18.759 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T23:31:18.759 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T23:31:18.759 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T23:31:18.759 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T23:31:18.759 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-08T23:31:18.760 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T23:31:18.761 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-08T23:31:18.761 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T23:31:18.761 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T23:31:18.761 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T23:31:18.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T23:31:18.821 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T23:31:18.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T23:31:18.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:31:18.911 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.42'), ('mon.c', '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]')] 2024-05-08T23:31:18.912 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T23:31:18.913 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:18.913 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:31:18.973 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.42 --addv c '[v2:172.21.15.42:3301,v1:172.21.15.42:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:19.072 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:19.072 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid 7b439b97-64c0-4aab-ac3b-1b32ed376bfb 2024-05-08T23:31:19.072 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = pacific 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 7b439b97-64c0-4aab-ac3b-1b32ed376bfb 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2024-05-08T23:31:19.072891+0000 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-08T23:31:19.072891+0000 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 16 (pacific) 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2024-05-08T23:31:19.073 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-08T23:31:19.074 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.b 2024-05-08T23:31:19.074 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] mon.c 2024-05-08T23:31:19.074 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T23:31:19.075 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T23:31:19.094 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7b439b97-64c0-4aab-ac3b-1b32ed376bfb... 2024-05-08T23:31:19.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:31:19.137 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:31:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-05-08T23:31:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790]" 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on skipped message = False 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 7b439b97-64c0-4aab-ac3b-1b32ed376bfb 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.196 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-05-08T23:31:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-05-08T23:31:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bdev async discard = True 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bdev enable discard = True 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = bitmap 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 1/20 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 1/20 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-05-08T23:31:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 4/10 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: filestore flush min = 0 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: mon client directed command retry = 5 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay max = 1 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay probability = 0.005 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay type = client mds 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject socket failures = 2500 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: osd blocked scrub grace period = 3600 2024-05-08T23:31:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: osd max markdown count = 1000 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.201 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:31:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-05-08T23:31:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: client_shutdown_timeout = 120 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e] 2024-05-08T23:31:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f] 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds balancer = 20 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds bal fragment size max = 10000 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds bal merge size = 5 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds bal split bits = 3 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds bal split size = 100 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds oft prefetch dirfrags = True 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2024-05-08T23:31:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: mds_dir_prefetch = True 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: mds_sleep_rank_change = 5000000.0 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: mon client directed command retry = 5 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay max = 1 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay probability = 0.005 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject delay type = client mds osd 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject socket failures = 2500 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-05-08T23:31:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-05-08T23:31:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T23:31:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = "172.21.15.33,172.21.15.42,[v2:172.21.15.42:3301,v1:172.21.15.42:6790]" 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on skipped message = False 2024-05-08T23:31:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 7b439b97-64c0-4aab-ac3b-1b32ed376bfb 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-05-08T23:31:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-05-08T23:31:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: bdev async discard = True 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: bdev enable discard = True 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore allocator = bitmap 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: filestore flush min = 0 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-05-08T23:31:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay max = 1 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay probability = 0.005 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay type = client mds 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 2500 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: osd blocked scrub grace period = 3600 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: osd max markdown count = 1000 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: osd op complaint time = 180 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-05-08T23:31:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-05-08T23:31:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-05-08T23:31:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: mon op complaint time = 120 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: client mount timeout = 600 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: client_shutdown_timeout = 120 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: debug client = 20 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: rados mon op timeout = 900 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: rados osd op timeout = 900 2024-05-08T23:31:19.217 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.b] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.c] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout:[mds] 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: debug mds = 20 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: debug mds balancer = 20 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-05-08T23:31:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: mds bal fragment size max = 10000 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds bal merge size = 5 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds bal split bits = 3 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds bal split size = 100 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds debug frag = True 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds debug scatterstat = True 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds oft prefetch dirfrags = True 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds op complaint time = 180 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds verify scatter = True 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds_dir_prefetch = True 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mds_sleep_rank_change = 5000000.0 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: mon client directed command retry = 5 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay max = 1 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay probability = 0.005 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject delay type = client mds osd 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: ms inject socket failures = 2500 2024-05-08T23:31:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: osd op complaint time = 180 2024-05-08T23:31:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: rados mon op timeout = 900 2024-05-08T23:31:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: rados osd op timeout = 900 2024-05-08T23:31:19.221 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T23:31:19.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T23:31:19.289 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T23:31:19.289 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:19.289 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T23:31:19.309 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:19.309 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T23:31:19.369 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:31:19.369 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:19.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:31:19.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:31:19.457 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:19.457 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:19.519 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:31:19.519 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:19.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T23:31:19.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T23:31:19.567 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:19.567 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:19.630 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T23:31:19.630 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T23:31:19.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:31:19.698 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T23:31:19.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:31:19.785 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T23:31:19.788 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T23:31:19.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:31:19.856 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T23:31:19.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T23:31:19.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:31:20.000 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T23:31:20.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T23:31:20.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-05-08T23:31:20.144 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-05-08T23:31:20.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-05-08T23:31:20.179 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:31:20.247 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T23:31:20.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T23:31:20.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:31:20.390 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T23:31:20.393 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T23:31:20.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-05-08T23:31:20.534 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-05-08T23:31:20.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-05-08T23:31:20.570 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T23:31:20.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:31:20.624 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T23:31:20.640 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T23:31:20.640 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T23:31:20.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T23:31:20.672 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:20.672 INFO:tasks.ceph:role: osd.0 2024-05-08T23:31:20.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:31:20.673 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:21.187 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-05-08T23:31:21.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-05-08T23:31:21.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T23:31:21.390 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T23:31:21.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T23:31:21.508 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:21.508 INFO:tasks.ceph:role: osd.1 2024-05-08T23:31:21.508 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:31:21.508 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:21.590 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:21.591 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:21.591 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:21.994 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-05-08T23:31:21.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-05-08T23:31:21.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T23:31:22.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T23:31:22.168 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T23:31:22.243 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:22.243 INFO:tasks.ceph:role: osd.2 2024-05-08T23:31:22.243 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:31:22.244 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:31:22.326 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:22.326 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:22.327 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:22.772 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-05-08T23:31:22.775 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-05-08T23:31:22.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T23:31:22.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T23:31:22.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T23:31:23.059 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:23.059 INFO:tasks.ceph:role: osd.3 2024-05-08T23:31:23.059 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:31:23.060 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:23.142 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:23.578 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-05-08T23:31:23.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-05-08T23:31:23.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T23:31:23.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T23:31:23.780 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.880+0000 7f72307897c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T23:31:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.881+0000 7f72307897c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T23:31:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.881+0000 7f72307897c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:31:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.881+0000 7f72307897c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:31:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.881+0000 7f72307897c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T23:31:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:23.881+0000 7f72307897c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T23:31:26.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T23:31:26.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.874+0000 7f515cf237c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T23:31:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.874+0000 7f515cf237c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.874+0000 7f515cf237c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.874+0000 7f515cf237c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.874+0000 7f515cf237c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T23:31:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:26.875+0000 7f515cf237c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T23:31:29.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T23:31:29.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.860+0000 7f6f2ed067c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T23:31:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.861+0000 7f6f2ed067c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T23:31:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.861+0000 7f6f2ed067c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:31:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.861+0000 7f6f2ed067c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:31:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.861+0000 7f6f2ed067c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T23:31:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:29.861+0000 7f6f2ed067c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T23:31:32.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T23:31:32.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.865+0000 7f24dda9c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T23:31:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.865+0000 7f24dda9c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T23:31:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.865+0000 7f24dda9c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:31:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.865+0000 7f24dda9c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:31:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.865+0000 7f24dda9c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T23:31:32.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:32.866+0000 7f24dda9c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T23:31:35.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T23:31:35.755 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T23:31:35.791 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:35.791 INFO:tasks.ceph:role: osd.4 2024-05-08T23:31:35.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:31:35.791 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:35.866 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:35.889 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-08T23:31:35.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-08T23:31:35.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T23:31:35.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T23:31:36.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T23:31:36.093 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:36.094 INFO:tasks.ceph:role: osd.5 2024-05-08T23:31:36.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:31:36.094 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:36.168 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:36.190 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-08T23:31:36.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-08T23:31:36.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T23:31:36.281 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T23:31:36.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T23:31:36.394 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:36.394 INFO:tasks.ceph:role: osd.6 2024-05-08T23:31:36.394 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:31:36.395 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:36.469 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:36.470 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:36.470 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:36.470 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:36.470 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:36.492 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-08T23:31:36.494 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-08T23:31:36.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T23:31:36.583 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T23:31:36.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T23:31:36.696 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T23:31:36.696 INFO:tasks.ceph:role: osd.7 2024-05-08T23:31:36.696 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:31:36.696 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T23:31:36.770 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-05-08T23:31:36.771 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T23:31:36.771 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-08T23:31:36.771 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T23:31:36.771 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T23:31:36.793 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-05-08T23:31:36.796 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-05-08T23:31:36.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T23:31:36.885 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T23:31:36.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.020+0000 7fd0c03887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T23:31:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.020+0000 7fd0c03887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T23:31:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.020+0000 7fd0c03887c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:31:37.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.020+0000 7fd0c03887c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:31:37.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.020+0000 7fd0c03887c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T23:31:37.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:37.021+0000 7fd0c03887c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T23:31:39.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T23:31:39.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T23:31:40.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:40.008+0000 7f146e91e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T23:31:42.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T23:31:42.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.007+0000 7f751e6e77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T23:31:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.008+0000 7f751e6e77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T23:31:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.008+0000 7f751e6e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:31:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.008+0000 7f751e6e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:31:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.008+0000 7f751e6e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T23:31:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:43.008+0000 7f751e6e77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T23:31:45.853 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T23:31:45.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.989+0000 7f5721ae57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T23:31:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.989+0000 7f5721ae57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T23:31:45.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.989+0000 7f5721ae57c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:31:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.989+0000 7f5721ae57c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:31:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.989+0000 7f5721ae57c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T23:31:45.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:45.990+0000 7f5721ae57c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T23:31:46.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:46.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:31:48.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T23:31:48.868 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T23:31:48.868 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:48.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T23:31:48.903 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:48.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T23:31:48.977 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:48.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T23:31:49.052 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-05-08T23:31:49.127 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T23:31:49.201 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T23:31:49.275 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T23:31:49.350 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T23:31:49.424 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T23:31:49.459 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T23:31:49.534 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T23:31:49.610 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-05-08T23:31:49.685 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T23:31:49.760 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.760 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T23:31:49.835 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T23:31:49.908 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:49.909 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T23:31:49.982 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:49.982 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T23:31:50.004 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T23:31:50.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:31:50.047 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBHCzxmKGqtKRAArIy006qS3ooKo+pT19OBQA== 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBHCzxmMzEdMxAAZx4sTTinU3/5XGpkmHpGuw== 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-05-08T23:31:50.077 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxmUygbABAALpey2Y1+AIyYR3JlUVDhtg== 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e] 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxm6AKtCBAA3mk/FO3wuxwcMvC3+GWcxg== 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBLCzxmjvCZNBAAxvgo9GG1OnhnyuF/sXvn8A== 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBOCzxmHWQ3NBAAiyf4eBmbLAJKGy43fZCTEg== 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-05-08T23:31:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBRCzxmCDdmMxAA5uVcTuFkCiQQ7aXN+kGfgw== 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBUCzxmYWKvMxAArMqmEfB6154RuYDIUahntA== 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBHCzxm+QfbLhAAtAXcAmvEHt0puF+/sE3l6Q== 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxmC0TYDhAABzAb6dn1XonF7ieGCpH1Uw== 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxmYrZQFxAA23dBs0Kbo50NlbyBu72gEg== 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f] 2024-05-08T23:31:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxmNMzmHxAAR5wzzZ5u+NbsnmxNPOglaQ== 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBZCzxmDUtDARAApoXytH4bt2hQXPYj5erCIQ== 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBcCzxmB2iGABAAVAyn8swq4BEAjHni1a59Sw== 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBfCzxmTCqEABAAEPwttn3iiqLjh3oybZ2BUA== 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBhCzxmuhgGOxAAaEN/ZxLRIwdMZdo3OOnjJw== 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-05-08T23:31:50.080 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBICzxmStxOJRAA2Vci5JUtGUWFEdqH11m/Mg== 2024-05-08T23:31:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.y] 2024-05-08T23:31:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBHCzxmKGqtKRAArIy006qS3ooKo+pT19OBQA== 2024-05-08T23:31:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-05-08T23:31:50.082 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBHCzxmMzEdMxAAZx4sTTinU3/5XGpkmHpGuw== 2024-05-08T23:31:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxmUygbABAALpey2Y1+AIyYR3JlUVDhtg== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxm6AKtCBAA3mk/FO3wuxwcMvC3+GWcxg== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBLCzxmjvCZNBAAxvgo9GG1OnhnyuF/sXvn8A== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBOCzxmHWQ3NBAAiyf4eBmbLAJKGy43fZCTEg== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBRCzxmCDdmMxAA5uVcTuFkCiQQ7aXN+kGfgw== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.3] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBUCzxmYWKvMxAArMqmEfB6154RuYDIUahntA== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBHCzxm+QfbLhAAtAXcAmvEHt0puF+/sE3l6Q== 2024-05-08T23:31:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxmC0TYDhAABzAb6dn1XonF7ieGCpH1Uw== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxmYrZQFxAA23dBs0Kbo50NlbyBu72gEg== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxmNMzmHxAAR5wzzZ5u+NbsnmxNPOglaQ== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.4] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBZCzxmDUtDARAApoXytH4bt2hQXPYj5erCIQ== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.5] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBcCzxmB2iGABAAVAyn8swq4BEAjHni1a59Sw== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.6] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBfCzxmTCqEABAAEPwttn3iiqLjh3oybZ2BUA== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.7] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBhCzxmuhgGOxAAaEN/ZxLRIwdMZdo3OOnjJw== 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-05-08T23:31:50.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBICzxmStxOJRAA2Vci5JUtGUWFEdqH11m/Mg== 2024-05-08T23:31:50.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:31:50.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:31:50.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.260 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.580 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:31:50.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T23:31:50.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T23:31:50.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:50.962 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:51.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:51.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:51.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:51.086 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T23:31:51.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:31:51.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T23:31:51.211 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T23:31:51.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T23:31:51.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:31:51.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T23:31:51.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T23:31:51.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:31:51.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T23:31:51.564 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T23:31:51.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T23:31:51.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T23:31:51.783 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:51.785 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T23:31:51.844 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T23:31:51.845 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:31:51.845 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:31:51.847 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:31:51.848 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T23:31:51.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T23:31:51.887 INFO:tasks.ceph.mon.b:Started 2024-05-08T23:31:51.887 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T23:31:51.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T23:31:51.890 INFO:tasks.ceph.mon.c:Started 2024-05-08T23:31:51.890 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T23:31:51.890 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T23:31:51.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T23:31:51.893 INFO:tasks.ceph.mgr.y:Started 2024-05-08T23:31:51.893 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T23:31:51.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T23:31:51.896 INFO:tasks.ceph.mgr.x:Started 2024-05-08T23:31:51.896 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T23:31:51.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T23:31:52.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.036+0000 7f55a1651700 1 Processor -- start 2024-05-08T23:31:52.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.036+0000 7f55a1651700 1 -- start start 2024-05-08T23:31:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f55a1651700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 0x7f559c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f55a1651700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c0cc340 con 0x7f559c154700 2024-05-08T23:31:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f55a1651700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c0caa30 con 0x7f559c0ca5c0 2024-05-08T23:31:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f55a1651700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c15a2a0 con 0x7f559c15dc50 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f559affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f559c154700 0x7f559c06d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f559affd700 1 -- 172.21.15.33:0/806510631 learned_addr learned my addr 172.21.15.33:0/806510631 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.037+0000 7f559b7fe700 1 --2- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 0x7f559c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.038+0000 7f559b7fe700 1 -- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 msgr2=0x7f559c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.038+0000 7f559b7fe700 1 -- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 msgr2=0x7f559c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.038+0000 7f559b7fe700 1 --2- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 0x7f559c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:31:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.038+0000 7f559b7fe700 1 --2- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 0x7f559c15e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:31:52.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.228+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475835511 0 0) 0x7f559c0cc340 con 0x7f559c154700 2024-05-08T23:31:52.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.228+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55880035b0 con 0x7f559c154700 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.228+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486796621 0 0) 0x7f559c0caa30 con 0x7f559c0ca5c0 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c0cc340 con 0x7f559c0ca5c0 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23477898 0 0) 0x7f559c0cc340 con 0x7f559c0ca5c0 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 msgr2=0x7f559c15e0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 --2- 172.21.15.33:0/806510631 >> v2:172.21.15.42:3300/0 conn(0x7f559c15dc50 0x7f559c15e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 >> v1:172.21.15.33:6789/0 conn(0x7f559c154700 legacy=0x7f559c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c15e5e0 con 0x7f559c0ca5c0 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f5584001670 con 0x7f559c0ca5c0 2024-05-08T23:31:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 >> v1:172.21.15.42:6789/0 conn(0x7f559c0ca5c0 legacy=0x7f559c0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 0x7f5588003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588004850 0x7f5588004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5588005210 0x7f558800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0015b0 con 0x7f5588003b90 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c15a2a0 con 0x7f5588004850 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558800c1c0 con 0x7f5588005210 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.229+0000 7f559affd700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 0x7f5588003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559a7fc700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588004850 0x7f5588004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559b7fe700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5588005210 0x7f558800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 -- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5588005210 msgr2=0x7f558800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5588005210 0x7f558800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 -- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588004850 msgr2=0x7f5588004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588004850 0x7f5588004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 -- 172.21.15.33:0/806510631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0cc340 con 0x7f5588003b90 2024-05-08T23:31:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559b7fe700 1 -- 172.21.15.33:0/806510631 reap_dead start 2024-05-08T23:31:52.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.230+0000 7f559affd700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 0x7f5588003f70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f558c00a7a0 tx=0x7f5588005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9873581b186497 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:52.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.232+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00a210 con 0x7f5588003b90 2024-05-08T23:31:52.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.232+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c007370 con 0x7f5588003b90 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.232+0000 7f5599ffb700 1 -- 172.21.15.33:0/806510631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c0076e0 con 0x7f5588003b90 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 -- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 msgr2=0x7f5588003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 0x7f5588003f70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f558c00a7a0 tx=0x7f5588005880 comp rx=0 tx=0).stop 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 -- 172.21.15.33:0/806510631 shutdown_connections 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 --2- 172.21.15.33:0/806510631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5588003b90 0x7f5588003f70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 -- 172.21.15.33:0/806510631 >> 172.21.15.33:0/806510631 conn(0x7f559c066cf0 msgr2=0x7f559c06ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.233+0000 7f55a1651700 1 -- 172.21.15.33:0/806510631 shutdown_connections 2024-05-08T23:31:52.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.234+0000 7f55a1651700 1 -- 172.21.15.33:0/806510631 wait complete. 2024-05-08T23:31:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.237+0000 7f55a1651700 1 Processor -- start 2024-05-08T23:31:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.237+0000 7f55a1651700 1 -- start start 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588003b90 0x7f559c1f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f559c0ce9b0 0x7f559c146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0d09f0 con 0x7f559c1452a0 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0ce560 con 0x7f5588003b90 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f55a1651700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0cda70 con 0x7f559c0ce9b0 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60832/0 (socket says 172.21.15.33:60832) 2024-05-08T23:31:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 -- 172.21.15.33:0/965666309 learned_addr learned my addr 172.21.15.33:0/965666309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559b7fe700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f559c0ce9b0 0x7f559c146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559affd700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588003b90 0x7f559c1f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 -- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f559c0ce9b0 msgr2=0x7f559c146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f559c0ce9b0 0x7f559c146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 -- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588003b90 msgr2=0x7f559c1f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588003b90 0x7f559c1f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.238+0000 7f559a7fc700 1 -- 172.21.15.33:0/965666309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c15d730 con 0x7f559c1452a0 2024-05-08T23:31:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f559a7fc700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f558400fb10 tx=0x7f558400fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55840050d0 con 0x7f559c1452a0 2024-05-08T23:31:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584005230 con 0x7f559c1452a0 2024-05-08T23:31:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584003680 con 0x7f559c1452a0 2024-05-08T23:31:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c0cdcc0 con 0x7f559c1452a0 2024-05-08T23:31:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c0d06b0 con 0x7f559c1452a0 2024-05-08T23:31:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f5584003ae0 con 0x7f559c1452a0 2024-05-08T23:31:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.239+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f558400dce0 con 0x7f559c1452a0 2024-05-08T23:31:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.240+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c0ca000 con 0x7f559c1452a0 2024-05-08T23:31:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.243+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f558400d560 con 0x7f559c1452a0 2024-05-08T23:31:52.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.327+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f559c075440 con 0x7f559c1452a0 2024-05-08T23:31:52.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.329+0000 7f55937fe700 1 -- 172.21.15.33:0/965666309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f558400d700 con 0x7f559c1452a0 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 msgr2=0x7f559c0cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f558400fb10 tx=0x7f558400fe20 comp rx=0 tx=0).stop 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 shutdown_connections 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f559c0ce9b0 0x7f559c146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5588003b90 0x7f559c1f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 --2- 172.21.15.33:0/965666309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c1452a0 0x7f559c0cb670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 >> 172.21.15.33:0/965666309 conn(0x7f559c066cf0 msgr2=0x7f559c152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:52.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 shutdown_connections 2024-05-08T23:31:52.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:52.330+0000 7f55a1651700 1 -- 172.21.15.33:0/965666309 wait complete. 2024-05-08T23:31:52.331 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-05-08T23:31:52.344 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T23:31:52.344 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T23:31:52.344 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:52.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T23:31:52.375 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:52.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T23:31:52.447 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:52.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T23:31:52.518 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:31:52.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T23:31:52.589 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:52.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T23:31:52.620 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:52.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T23:31:52.692 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:52.692 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T23:31:52.720 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:52.721+0000 7f8e9bcbd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:31:52.761 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-05-08T23:31:52.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T23:31:52.792 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:52.792+0000 7f9cfacc2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:31:52.810 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:52.811+0000 7f8e9bcbd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:31:52.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new d43d2aca-6883-419c-8beb-8caa4f3e5453 0 2024-05-08T23:31:52.884 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:52.884+0000 7f9cfacc2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:31:52.894 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:52.895+0000 7f8e9bcbd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:31:52.969 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:52.970+0000 7f9cfacc2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:31:52.975 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:52.975+0000 7f8e9bcbd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:31:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.997+0000 7f80b1801700 1 Processor -- start 2024-05-08T23:31:52.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.997+0000 7f80b1801700 1 -- start start 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80b1801700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80b1801700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80ac0c9a60 0x7f80ac0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80b1801700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac070450 con 0x7f80ac152350 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80b1801700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ac0c2e90 con 0x7f80ac0c9a60 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80b1801700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac070730 con 0x7f80ac0c38a0 2024-05-08T23:31:53.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80aa7fc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80ac0c9a60 0x7f80ac0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f80ac0c9a60 0x7f80ac0c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60546/0 (socket says 172.21.15.42:60546) 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80aa7fc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44164/0 (socket says 172.21.15.42:44164) 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 -- 172.21.15.42:0/2239794586 learned_addr learned my addr 172.21.15.42:0/2239794586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 -- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 msgr2=0x7f80ac0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 -- 172.21.15.42:0/2239794586 >> v1:172.21.15.42:6789/0 conn(0x7f80ac152350 legacy=0x7f80ac158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 -- 172.21.15.42:0/2239794586 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac070a10 con 0x7f80ac0c9a60 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80aa7fc700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:52.999+0000 7f80ab7fe700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac0c9a60 0x7f80ac0c85a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009e50 tx=0x7f80a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2cc4ba2b9e181b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80a9ffb700 1 -- 172.21.15.42:0/2239794586 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400e070 con 0x7f80ac0c9a60 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80a9ffb700 1 -- 172.21.15.42:0/2239794586 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a400b910 con 0x7f80ac0c9a60 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80a9ffb700 1 -- 172.21.15.42:0/2239794586 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400bcc0 con 0x7f80ac0c9a60 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 -- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac0c9a60 msgr2=0x7f80ac0c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac0c9a60 0x7f80ac0c85a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009e50 tx=0x7f80a400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 -- 172.21.15.42:0/2239794586 shutdown_connections 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac0c9a60 0x7f80ac0c85a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 --2- 172.21.15.42:0/2239794586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 -- 172.21.15.42:0/2239794586 >> 172.21.15.42:0/2239794586 conn(0x7f80ac0666c0 msgr2=0x7f80ac066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 -- 172.21.15.42:0/2239794586 shutdown_connections 2024-05-08T23:31:53.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.000+0000 7f80b1801700 1 -- 172.21.15.42:0/2239794586 wait complete. 2024-05-08T23:31:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.002+0000 7f80b1801700 1 Processor -- start 2024-05-08T23:31:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.002+0000 7f80b1801700 1 -- start start 2024-05-08T23:31:53.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac0c9a60 0x7f80ac146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 0x7f80ac146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac144cf0 con 0x7f80ac0c9a60 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac1f0cf0 con 0x7f80ac152350 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80b1801700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac1f0170 con 0x7f80ac0c38a0 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80ab7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 0x7f80ac146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44178/0 (socket says 172.21.15.42:44178) 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 -- 172.21.15.42:0/1632900364 learned_addr learned my addr 172.21.15.42:0/1632900364 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aa7fc700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac0c9a60 0x7f80ac146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 -- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 msgr2=0x7f80ac146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 0x7f80ac146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 -- 172.21.15.42:0/1632900364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac0c9a60 msgr2=0x7f80ac146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac0c9a60 0x7f80ac146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80aaffd700 1 -- 172.21.15.42:0/1632900364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00094a0 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.003+0000 7f80ab7fe700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 0x7f80ac146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80aaffd700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c480 tx=0x7f809c00c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0052d0 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c005430 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a4009b30 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0038c0 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac1f0730 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f809c003d20 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.004+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f809c01c780 con 0x7f80ac0c38a0 2024-05-08T23:31:53.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.005+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac075460 con 0x7f80ac0c38a0 2024-05-08T23:31:53.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.007+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f809c017070 con 0x7f80ac0c38a0 2024-05-08T23:31:53.052 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.053+0000 7f9cfacc2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:31:53.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.098+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d43d2aca-6883-419c-8beb-8caa4f3e5453", "id": 0} v 0) v1 -- 0x7f80ac06bc10 con 0x7f80ac0c38a0 2024-05-08T23:31:53.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.102+0000 7f80937fe700 1 -- 172.21.15.42:0/1632900364 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d43d2aca-6883-419c-8beb-8caa4f3e5453", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f809c00ce50 con 0x7f80ac0c38a0 2024-05-08T23:31:53.101 INFO:teuthology.orchestra.run.smithi042.stdout:0 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 msgr2=0x7f80ac140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c480 tx=0x7f809c00c790 comp rx=0 tx=0).stop 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 shutdown_connections 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80ac152350 0x7f80ac146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80ac0c38a0 0x7f80ac140ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 --2- 172.21.15.42:0/1632900364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ac0c9a60 0x7f80ac146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 >> 172.21.15.42:0/1632900364 conn(0x7f80ac0666c0 msgr2=0x7f80ac066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 shutdown_connections 2024-05-08T23:31:53.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.103+0000 7f80b1801700 1 -- 172.21.15.42:0/1632900364 wait complete. 2024-05-08T23:31:53.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new c8d8fcfd-af87-49c6-bc12-c769b062cb0b 1 2024-05-08T23:31:53.172 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.172+0000 7f8e9bcbd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:31:53.253 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.253+0000 7f9cfacc2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:31:53.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.293+0000 7f98f8a34700 1 Processor -- start 2024-05-08T23:31:53.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.293+0000 7f98f8a34700 1 -- start start 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f8a34700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98f40caad0 0x7f98f40cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f8a34700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f8a34700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f415b350 con 0x7f98f4154970 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f8a34700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f415e8a0 con 0x7f98f40caad0 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f8a34700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f415a5d0 con 0x7f98f40cbbe0 2024-05-08T23:31:53.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f2d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f2d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44180/0 (socket says 172.21.15.42:44180) 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f2d9d700 1 -- 172.21.15.42:0/2534055990 learned_addr learned my addr 172.21.15.42:0/2534055990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 --2- 172.21.15.42:0/2534055990 >> v2:172.21.15.33:3300/0 conn(0x7f98f40caad0 0x7f98f40cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 -- 172.21.15.42:0/2534055990 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 msgr2=0x7f98f415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 -- 172.21.15.42:0/2534055990 >> v1:172.21.15.33:6789/0 conn(0x7f98f4154970 legacy=0x7f98f406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 -- 172.21.15.42:0/2534055990 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f415e5c0 con 0x7f98f40caad0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98f2d9d700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.294+0000 7f98e9d9b700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40caad0 0x7f98f40cb6a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009840 tx=0x7f98dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f926ec5670346635 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f1d9b700 1 -- 172.21.15.42:0/2534055990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc011830 con 0x7f98f40caad0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f1d9b700 1 -- 172.21.15.42:0/2534055990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc011990 con 0x7f98f40caad0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f1d9b700 1 -- 172.21.15.42:0/2534055990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc011d20 con 0x7f98f40caad0 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 -- 172.21.15.42:0/2534055990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40caad0 msgr2=0x7f98f40cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40caad0 0x7f98f40cb6a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009840 tx=0x7f98dc009b50 comp rx=0 tx=0).stop 2024-05-08T23:31:53.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 -- 172.21.15.42:0/2534055990 shutdown_connections 2024-05-08T23:31:53.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f40cbbe0 0x7f98f415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 --2- 172.21.15.42:0/2534055990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40caad0 0x7f98f40cb6a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 -- 172.21.15.42:0/2534055990 >> 172.21.15.42:0/2534055990 conn(0x7f98f406cd80 msgr2=0x7f98f406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 -- 172.21.15.42:0/2534055990 shutdown_connections 2024-05-08T23:31:53.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.295+0000 7f98f8a34700 1 -- 172.21.15.42:0/2534055990 wait complete. 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.297+0000 7f98f8a34700 1 Processor -- start 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.297+0000 7f98f8a34700 1 -- start start 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 0x7f98f40c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40cbbe0 0x7f98f414c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f41f0230 con 0x7f98f40cbbe0 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f415b590 con 0x7f98f40caad0 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f8a34700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f415b7e0 con 0x7f98f4154970 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f2d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f2d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44188/0 (socket says 172.21.15.42:44188) 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f2d9d700 1 -- 172.21.15.42:0/1062932779 learned_addr learned my addr 172.21.15.42:0/1062932779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 0x7f98f40c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98e9d9b700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40cbbe0 0x7f98f414c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 -- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 msgr2=0x7f98f40ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 -- 172.21.15.42:0/1062932779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40cbbe0 msgr2=0x7f98f414c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40cbbe0 0x7f98f414c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 -- 172.21.15.42:0/1062932779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009b30 con 0x7f98f40caad0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f2d9d700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.298+0000 7f98f259c700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 0x7f98f40c3170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98e4000f60 tx=0x7f98e400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.299+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40052d0 con 0x7f98f40caad0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.299+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4005430 con 0x7f98f40caad0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.299+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40038a0 con 0x7f98f40caad0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.299+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0094a0 con 0x7f98f40caad0 2024-05-08T23:31:53.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.299+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f41f0440 con 0x7f98f40caad0 2024-05-08T23:31:53.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.300+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f98e4003d00 con 0x7f98f40caad0 2024-05-08T23:31:53.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.300+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f406c750 con 0x7f98f40caad0 2024-05-08T23:31:53.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.300+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f98e401cd80 con 0x7f98f40caad0 2024-05-08T23:31:53.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.302+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f98e4017070 con 0x7f98f40caad0 2024-05-08T23:31:53.361 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.362+0000 7f8e9bcbd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:31:53.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.395+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c8d8fcfd-af87-49c6-bc12-c769b062cb0b", "id": 1} v 0) v1 -- 0x7f98f4075460 con 0x7f98f40caad0 2024-05-08T23:31:53.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.403+0000 7f98eb7fe700 1 -- 172.21.15.42:0/1062932779 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8d8fcfd-af87-49c6-bc12-c769b062cb0b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98e401c440 con 0x7f98f40caad0 2024-05-08T23:31:53.402 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-05-08T23:31:53.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 msgr2=0x7f98f40c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 0x7f98f40c3170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98e4000f60 tx=0x7f98e400c6e0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 shutdown_connections 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98f4154970 0x7f98f40ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f40caad0 0x7f98f40c3170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 --2- 172.21.15.42:0/1062932779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f40cbbe0 0x7f98f414c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 >> 172.21.15.42:0/1062932779 conn(0x7f98f406cd80 msgr2=0x7f98f414ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 shutdown_connections 2024-05-08T23:31:53.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.404+0000 7f98f8a34700 1 -- 172.21.15.42:0/1062932779 wait complete. 2024-05-08T23:31:53.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new 35734e60-7ecf-4a32-a934-4a9be2857016 2 2024-05-08T23:31:53.451 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.451+0000 7f9cfacc2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:31:53.487 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.488+0000 7f8e9bcbd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:31:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.553+0000 7ff45c2db700 1 Processor -- start 2024-05-08T23:31:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.553+0000 7ff45c2db700 1 -- start start 2024-05-08T23:31:53.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45c2db700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4540ce150 0x7ff454153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45c2db700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4540c9c30 0x7ff454154510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45c2db700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45415e6e0 con 0x7ff4540cdd70 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45c2db700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff454154a50 con 0x7ff4540c9c30 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45c2db700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540c9090 con 0x7ff4540ce150 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff459876700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4540ce150 0x7ff454153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff459876700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff4540ce150 0x7ff454153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60574/0 (socket says 172.21.15.42:60574) 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff459876700 1 -- 172.21.15.42:0/3269123587 learned_addr learned my addr 172.21.15.42:0/3269123587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.554+0000 7ff45a878700 1 --2- 172.21.15.42:0/3269123587 >> v2:172.21.15.33:3300/0 conn(0x7ff4540c9c30 0x7ff454154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459876700 1 -- 172.21.15.42:0/3269123587 >> v2:172.21.15.33:3300/0 conn(0x7ff4540c9c30 msgr2=0x7ff454154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459876700 1 --2- 172.21.15.42:0/3269123587 >> v2:172.21.15.33:3300/0 conn(0x7ff4540c9c30 0x7ff454154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459876700 1 -- 172.21.15.42:0/3269123587 >> v1:172.21.15.42:6789/0 conn(0x7ff4540cdd70 legacy=0x7ff4540c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459876700 1 -- 172.21.15.42:0/3269123587 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540c3b60 con 0x7ff4540ce150 2024-05-08T23:31:53.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459876700 1 --2- 172.21.15.42:0/3269123587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff454153fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4440098f0 tx=0x7ff444009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35f3246d04246d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459075700 1 -- 172.21.15.42:0/3269123587 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011830 con 0x7ff4540ce150 2024-05-08T23:31:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459075700 1 -- 172.21.15.42:0/3269123587 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444011990 con 0x7ff4540ce150 2024-05-08T23:31:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.555+0000 7ff459075700 1 -- 172.21.15.42:0/3269123587 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011d20 con 0x7ff4540ce150 2024-05-08T23:31:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 -- 172.21.15.42:0/3269123587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 msgr2=0x7ff454153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 --2- 172.21.15.42:0/3269123587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff454153fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4440098f0 tx=0x7ff444009c00 comp rx=0 tx=0).stop 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 -- 172.21.15.42:0/3269123587 shutdown_connections 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 --2- 172.21.15.42:0/3269123587 >> v2:172.21.15.33:3300/0 conn(0x7ff4540c9c30 0x7ff454154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 --2- 172.21.15.42:0/3269123587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff454153fd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 -- 172.21.15.42:0/3269123587 >> 172.21.15.42:0/3269123587 conn(0x7ff4540666c0 msgr2=0x7ff454066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 -- 172.21.15.42:0/3269123587 shutdown_connections 2024-05-08T23:31:53.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.556+0000 7ff45c2db700 1 -- 172.21.15.42:0/3269123587 wait complete. 2024-05-08T23:31:53.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.557+0000 7ff45c2db700 1 Processor -- start 2024-05-08T23:31:53.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.557+0000 7ff45c2db700 1 -- start start 2024-05-08T23:31:53.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 0x7ff45415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541587c0 con 0x7ff4540c9c30 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454158be0 con 0x7ff4540ce150 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45c2db700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff454158140 con 0x7ff4540cdd70 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45a878700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff459876700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45a878700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60586/0 (socket says 172.21.15.42:60586) 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff459876700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44202/0 (socket says 172.21.15.42:44202) 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45a878700 1 -- 172.21.15.42:0/4115519075 learned_addr learned my addr 172.21.15.42:0/4115519075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.558+0000 7ff45a077700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 0x7ff45415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 -- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 msgr2=0x7ff4540c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 -- 172.21.15.42:0/4115519075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 msgr2=0x7ff45415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 0x7ff45415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 -- 172.21.15.42:0/4115519075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444009550 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a077700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 0x7ff45415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff459876700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45a878700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00efc0 tx=0x7ff44c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c009ea0 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c00f040 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00ced0 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff454159800 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.559+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4541583e0 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.560+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff44c014780 con 0x7ff4540ce150 2024-05-08T23:31:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.560+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7ff44c013800 con 0x7ff4540ce150 2024-05-08T23:31:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.560+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4540c40c0 con 0x7ff4540ce150 2024-05-08T23:31:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.563+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff44c018020 con 0x7ff4540ce150 2024-05-08T23:31:53.581 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.581+0000 7f9cfacc2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:31:53.595 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.596+0000 7f8e9bcbd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:31:53.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.653+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35734e60-7ecf-4a32-a934-4a9be2857016", "id": 2} v 0) v1 -- 0x7ff454158900 con 0x7ff4540ce150 2024-05-08T23:31:53.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.661+0000 7ff44affd700 1 -- 172.21.15.42:0/4115519075 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35734e60-7ecf-4a32-a934-4a9be2857016", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff454158900 con 0x7ff4540ce150 2024-05-08T23:31:53.660 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-05-08T23:31:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 msgr2=0x7ff4541f3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00efc0 tx=0x7ff44c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 shutdown_connections 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4540ce150 0x7ff4541f3060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4540c9c30 0x7ff45415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 --2- 172.21.15.42:0/4115519075 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4540cdd70 0x7ff4540c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 >> 172.21.15.42:0/4115519075 conn(0x7ff4540666c0 msgr2=0x7ff4540ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 shutdown_connections 2024-05-08T23:31:53.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.662+0000 7ff45c2db700 1 -- 172.21.15.42:0/4115519075 wait complete. 2024-05-08T23:31:53.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new b62de6c1-97fe-4574-9736-23c4c7a2ac56 3 2024-05-08T23:31:53.682 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.683+0000 7f8e9bcbd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:31:53.690 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.690+0000 7f9cfacc2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:31:53.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:53.779+0000 7f9cfacc2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:31:53.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.824+0000 7f724a7a4700 1 Processor -- start 2024-05-08T23:31:53.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.824+0000 7f724a7a4700 1 -- start start 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f724a7a4700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f72440ce150 0x7f7244153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f724a7a4700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72440c9c30 0x7f7244154510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f724a7a4700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724415e6e0 con 0x7f72440cdd70 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f724a7a4700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7244154a50 con 0x7f72440ce150 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f724a7a4700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72440c9090 con 0x7f72440c9c30 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72440c9c30 0x7f7244154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f72440c9c30 0x7f7244154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60600/0 (socket says 172.21.15.42:60600) 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 -- 172.21.15.42:0/1092219888 learned_addr learned my addr 172.21.15.42:0/1092219888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f72437fe700 1 --2- 172.21.15.42:0/1092219888 >> v2:172.21.15.33:3300/0 conn(0x7f72440ce150 0x7f7244153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 -- 172.21.15.42:0/1092219888 >> v2:172.21.15.33:3300/0 conn(0x7f72440ce150 msgr2=0x7f7244153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 --2- 172.21.15.42:0/1092219888 >> v2:172.21.15.33:3300/0 conn(0x7f72440ce150 0x7f7244153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 -- 172.21.15.42:0/1092219888 >> v1:172.21.15.42:6789/0 conn(0x7f72440cdd70 legacy=0x7f72440c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 -- 172.21.15.42:0/1092219888 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72440c3b60 con 0x7f72440c9c30 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.825+0000 7f7248d41700 1 --2- 172.21.15.42:0/1092219888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440c9c30 0x7f7244154510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e7205dc265341 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f7242ffd700 1 -- 172.21.15.42:0/1092219888 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00e070 con 0x7f72440c9c30 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f7242ffd700 1 -- 172.21.15.42:0/1092219888 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c00b8d0 con 0x7f72440c9c30 2024-05-08T23:31:53.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f7242ffd700 1 -- 172.21.15.42:0/1092219888 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00bc80 con 0x7f72440c9c30 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 -- 172.21.15.42:0/1092219888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440c9c30 msgr2=0x7f7244154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 --2- 172.21.15.42:0/1092219888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440c9c30 0x7f7244154510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f723c009e50 tx=0x7f723c00b330 comp rx=0 tx=0).stop 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 -- 172.21.15.42:0/1092219888 shutdown_connections 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 --2- 172.21.15.42:0/1092219888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440c9c30 0x7f7244154510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 --2- 172.21.15.42:0/1092219888 >> v2:172.21.15.33:3300/0 conn(0x7f72440ce150 0x7f7244153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 -- 172.21.15.42:0/1092219888 >> 172.21.15.42:0/1092219888 conn(0x7f72440666c0 msgr2=0x7f7244066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 -- 172.21.15.42:0/1092219888 shutdown_connections 2024-05-08T23:31:53.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.826+0000 7f724a7a4700 1 -- 172.21.15.42:0/1092219888 wait complete. 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.828+0000 7f724a7a4700 1 Processor -- start 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.828+0000 7f724a7a4700 1 -- start start 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 0x7f72441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72441f09e0 con 0x7f72440cdd70 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72441efb70 con 0x7f72440ce150 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f724a7a4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72441efd60 con 0x7f72440c9c30 2024-05-08T23:31:53.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7243fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7243fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44210/0 (socket says 172.21.15.42:44210) 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60602/0 (socket says 172.21.15.42:60602) 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7243fff700 1 -- 172.21.15.42:0/2294381330 learned_addr learned my addr 172.21.15.42:0/2294381330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f72437fe700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 0x7f72441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 -- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 msgr2=0x7f724415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 -- 172.21.15.42:0/2294381330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 msgr2=0x7f72441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 0x7f72441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 -- 172.21.15.42:0/2294381330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c009b30 con 0x7f72440ce150 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7243fff700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f72437fe700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 0x7f72441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:53.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.829+0000 7f7248d41700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f723c015000 tx=0x7f723c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c004800 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c0092e0 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c01c680 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72441f0040 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72441f0600 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f723c01cae0 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.830+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f723c025450 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.831+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72440c40c0 con 0x7f72440ce150 2024-05-08T23:31:53.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.833+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f723c043dc0 con 0x7f72440ce150 2024-05-08T23:31:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.925+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b62de6c1-97fe-4574-9736-23c4c7a2ac56", "id": 3} v 0) v1 -- 0x7f7244075460 con 0x7f72440ce150 2024-05-08T23:31:53.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.934+0000 7f7240ff9700 1 -- 172.21.15.42:0/2294381330 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b62de6c1-97fe-4574-9736-23c4c7a2ac56", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f723c032030 con 0x7f72440ce150 2024-05-08T23:31:53.934 INFO:teuthology.orchestra.run.smithi042.stdout:3 2024-05-08T23:31:53.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 msgr2=0x7f7244158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f723c015000 tx=0x7f723c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 shutdown_connections 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72440ce150 0x7f7244158d10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f72440c9c30 0x7f724415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 --2- 172.21.15.42:0/2294381330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72440cdd70 0x7f72441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 >> 172.21.15.42:0/2294381330 conn(0x7f72440666c0 msgr2=0x7f72440caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 shutdown_connections 2024-05-08T23:31:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:53.935+0000 7f724a7a4700 1 -- 172.21.15.42:0/2294381330 wait complete. 2024-05-08T23:31:53.944 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:53.945+0000 7f8e9bcbd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:31:53.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new 081cd8bf-a960-4db7-9b4e-028fc22b43ce 4 2024-05-08T23:31:54.024 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:54.025+0000 7f8e9bcbd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:31:54.046 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:54.046+0000 7f9cfacc2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:31:54.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.120+0000 7ff52d073700 1 Processor -- start 2024-05-08T23:31:54.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.120+0000 7ff52d073700 1 -- start start 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff52d073700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff5280cbef0 0x7ff528150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff52d073700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5280c3580 con 0x7ff5280cc5d0 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff52d073700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5280c2fa0 con 0x7ff5281572c0 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff52d073700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff528157db0 con 0x7ff5280cbef0 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff526d9d700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff5281572c0 0x7ff5280c3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51754/0 (socket says 172.21.15.42:51754) 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.121+0000 7ff526d9d700 1 -- 172.21.15.42:0/1702367025 learned_addr learned my addr 172.21.15.42:0/1702367025 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-08T23:31:54.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff52759e700 1 --2- 172.21.15.42:0/1702367025 >> v2:172.21.15.33:3300/0 conn(0x7ff5280cbef0 0x7ff528150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086989663 0 0) 0x7ff5280c2fa0 con 0x7ff5281572c0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5100035b0 con 0x7ff5281572c0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753097476 0 0) 0x7ff5280c3580 con 0x7ff5280cc5d0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5280c2fa0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff52759e700 1 -- 172.21.15.42:0/1702367025 >> v1:172.21.15.42:6789/0 conn(0x7ff5281572c0 legacy=0x7ff5280c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff52759e700 1 -- 172.21.15.42:0/1702367025 >> v1:172.21.15.33:6789/0 conn(0x7ff5280cc5d0 legacy=0x7ff5281576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff52759e700 1 -- 172.21.15.42:0/1702367025 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280ccaa0 con 0x7ff5280cbef0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff52759e700 1 --2- 172.21.15.42:0/1702367025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528150dc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff520009d90 tx=0x7ff52000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f9bdd48858684b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52000e070 con 0x7ff5280cbef0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52000bb20 con 0x7ff5280cbef0 2024-05-08T23:31:54.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.122+0000 7ff525d9b700 1 -- 172.21.15.42:0/1702367025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520012430 con 0x7ff5280cbef0 2024-05-08T23:31:54.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 -- 172.21.15.42:0/1702367025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 msgr2=0x7ff528150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 --2- 172.21.15.42:0/1702367025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528150dc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff520009d90 tx=0x7ff52000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:54.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 -- 172.21.15.42:0/1702367025 shutdown_connections 2024-05-08T23:31:54.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 --2- 172.21.15.42:0/1702367025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528150dc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 -- 172.21.15.42:0/1702367025 >> 172.21.15.42:0/1702367025 conn(0x7ff52806eb10 msgr2=0x7ff52806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 -- 172.21.15.42:0/1702367025 shutdown_connections 2024-05-08T23:31:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.123+0000 7ff52d073700 1 -- 172.21.15.42:0/1702367025 wait complete. 2024-05-08T23:31:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.125+0000 7ff52d073700 1 Processor -- start 2024-05-08T23:31:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.125+0000 7ff52d073700 1 -- start start 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 0x7ff528155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff528155040 con 0x7ff5280cbef0 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280c5470 con 0x7ff5281572c0 2024-05-08T23:31:54.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52d073700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280c5750 con 0x7ff5280cc5d0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52759e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52759e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60618/0 (socket says 172.21.15.42:60618) 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52759e700 1 -- 172.21.15.42:0/4001346084 learned_addr learned my addr 172.21.15.42:0/4001346084 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 0x7ff528155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff526d9d700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 -- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 msgr2=0x7ff528159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 -- 172.21.15.42:0/4001346084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 msgr2=0x7ff528155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 -- 172.21.15.42:0/4001346084 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520009a70 con 0x7ff5280cc5d0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52759e700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff52659c700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 0x7ff528155ff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c450 tx=0x7ff51c00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.126+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0052d0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c005430 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5281552b0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0038c0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52814f390 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff51c003d20 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.127+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff51c011e00 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.128+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5281557d0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.130+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff5281557d0 con 0x7ff5280cc5d0 2024-05-08T23:31:54.131 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:54.132+0000 7f9cfacc2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:31:54.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.269+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd new", "uuid": "081cd8bf-a960-4db7-9b4e-028fc22b43ce", "id": 4} v 0) v1 -- 0x7ff528075460 con 0x7ff5280cc5d0 2024-05-08T23:31:54.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.277+0000 7ff50f7fe700 1 -- 172.21.15.42:0/4001346084 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "081cd8bf-a960-4db7-9b4e-028fc22b43ce", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff51c017070 con 0x7ff5280cc5d0 2024-05-08T23:31:54.277 INFO:teuthology.orchestra.run.smithi042.stdout:4 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 msgr2=0x7ff528155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 0x7ff528155ff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c450 tx=0x7ff51c00c760 comp rx=0 tx=0).stop 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 shutdown_connections 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5281572c0 0x7ff528159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5280cbef0 0x7ff528155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 --2- 172.21.15.42:0/4001346084 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff5280cc5d0 0x7ff528155ff0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 >> 172.21.15.42:0/4001346084 conn(0x7ff52806eb10 msgr2=0x7ff52806d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 shutdown_connections 2024-05-08T23:31:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.278+0000 7ff52d073700 1 -- 172.21.15.42:0/4001346084 wait complete. 2024-05-08T23:31:54.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new 837c0f59-424f-4f17-8b56-e1e548e855f6 5 2024-05-08T23:31:54.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.426+0000 7fd6a35fd700 1 Processor -- start 2024-05-08T23:31:54.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.426+0000 7fd6a35fd700 1 -- start start 2024-05-08T23:31:54.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a35fd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a35fd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 0x7fd69c06aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a35fd700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c15d870 con 0x7fd69c154460 2024-05-08T23:31:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a35fd700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c071150 con 0x7fd69c15ad80 2024-05-08T23:31:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a35fd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0c7e30 con 0x7fd69c06f810 2024-05-08T23:31:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a0b98700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a0b98700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60628/0 (socket says 172.21.15.42:60628) 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a1b9a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 0x7fd69c06aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a0b98700 1 -- 172.21.15.42:0/3399748771 learned_addr learned my addr 172.21.15.42:0/3399748771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a1b9a700 1 -- 172.21.15.42:0/3399748771 >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 msgr2=0x7fd69c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a1b9a700 1 --2- 172.21.15.42:0/3399748771 >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a1b9a700 1 -- 172.21.15.42:0/3399748771 >> v1:172.21.15.42:6789/0 conn(0x7fd69c154460 legacy=0x7fd69c15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a1b9a700 1 -- 172.21.15.42:0/3399748771 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c15b220 con 0x7fd69c06f810 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.427+0000 7fd6a0b98700 1 --2- 172.21.15.42:0/3399748771 >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a1b9a700 1 --2- 172.21.15.42:0/3399748771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 0x7fd69c06aed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd694009d90 tx=0x7fd69400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9b27818a14124c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd693fff700 1 -- 172.21.15.42:0/3399748771 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69400e070 con 0x7fd69c06f810 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd693fff700 1 -- 172.21.15.42:0/3399748771 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69400bb20 con 0x7fd69c06f810 2024-05-08T23:31:54.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd693fff700 1 -- 172.21.15.42:0/3399748771 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694012430 con 0x7fd69c06f810 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 -- 172.21.15.42:0/3399748771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 msgr2=0x7fd69c06aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/3399748771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 0x7fd69c06aed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd694009d90 tx=0x7fd69400b330 comp rx=0 tx=0).stop 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 -- 172.21.15.42:0/3399748771 shutdown_connections 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/3399748771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c06f810 0x7fd69c06aed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/3399748771 >> v2:172.21.15.42:3300/0 conn(0x7fd69c15ad80 0x7fd69c06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 -- 172.21.15.42:0/3399748771 >> 172.21.15.42:0/3399748771 conn(0x7fd69c06e990 msgr2=0x7fd69c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 -- 172.21.15.42:0/3399748771 shutdown_connections 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.428+0000 7fd6a35fd700 1 -- 172.21.15.42:0/3399748771 wait complete. 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 Processor -- start 2024-05-08T23:31:54.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 -- start start 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c154460 0x7fd69c06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 0x7fd69c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c155b30 con 0x7fd69c15ad80 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c15cab0 con 0x7fd69c06f810 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.430+0000 7fd6a35fd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c15d630 con 0x7fd69c154460 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60632/0 (socket says 172.21.15.42:60632) 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 -- 172.21.15.42:0/438915557 learned_addr learned my addr 172.21.15.42:0/438915557 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a0b98700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c154460 0x7fd69c06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1b9a700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 0x7fd69c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 -- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c154460 msgr2=0x7fd69c06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c154460 0x7fd69c06ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 -- 172.21.15.42:0/438915557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 msgr2=0x7fd69c1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 0x7fd69c1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 -- 172.21.15.42:0/438915557 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009550 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1b9a700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 0x7fd69c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a1399700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c450 tx=0x7fd69800c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980052d0 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd698005430 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd694009a70 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.431+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980038c0 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.432+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd698003d20 con 0x7fd69c06f810 2024-05-08T23:31:54.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.432+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c15cd00 con 0x7fd69c06f810 2024-05-08T23:31:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.432+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd698011db0 con 0x7fd69c06f810 2024-05-08T23:31:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.433+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c075460 con 0x7fd69c06f810 2024-05-08T23:31:54.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.435+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fd698017070 con 0x7fd69c06f810 2024-05-08T23:31:54.448 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:54.449+0000 7f8e9bcbd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:31:54.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.527+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "837c0f59-424f-4f17-8b56-e1e548e855f6", "id": 5} v 0) v1 -- 0x7fd69c0b2e10 con 0x7fd69c06f810 2024-05-08T23:31:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.535+0000 7fd691ffb700 1 -- 172.21.15.42:0/438915557 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "837c0f59-424f-4f17-8b56-e1e548e855f6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd698011590 con 0x7fd69c06f810 2024-05-08T23:31:54.535 INFO:teuthology.orchestra.run.smithi042.stdout:5 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 msgr2=0x7fd69c14ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c450 tx=0x7fd69800c760 comp rx=0 tx=0).stop 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 shutdown_connections 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd69c15ad80 0x7fd69c1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd69c06f810 0x7fd69c14ac10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 --2- 172.21.15.42:0/438915557 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd69c154460 0x7fd69c06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 >> 172.21.15.42:0/438915557 conn(0x7fd69c06e990 msgr2=0x7fd69c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 shutdown_connections 2024-05-08T23:31:54.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.536+0000 7fd6a35fd700 1 -- 172.21.15.42:0/438915557 wait complete. 2024-05-08T23:31:54.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new e7c0c6d9-0fb2-44c8-90eb-59428c93016e 6 2024-05-08T23:31:54.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.686+0000 7fed96cf1700 1 Processor -- start 2024-05-08T23:31:54.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.686+0000 7fed96cf1700 1 -- start start 2024-05-08T23:31:54.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.686+0000 7fed96cf1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 0x7fed90163a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed96cf1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9006e3c0 con 0x7fed901522f0 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed96cf1700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9006ecf0 con 0x7fed900c2e90 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed96cf1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed900c9cf0 con 0x7fed900c92d0 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed9528e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 0x7fed90163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed8ffff700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fed900c2e90 0x7fed900ce3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.42:51778/0 (socket says 172.21.15.42:51778) 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed8ffff700 1 -- 172.21.15.42:0/530009760 learned_addr learned my addr 172.21.15.42:0/530009760 (peer_addr_for_me v1:172.21.15.42:0/0) 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed9528e700 1 -- 172.21.15.42:0/530009760 >> v1:172.21.15.42:6789/0 conn(0x7fed900c2e90 legacy=0x7fed900ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed9528e700 1 -- 172.21.15.42:0/530009760 >> v1:172.21.15.33:6789/0 conn(0x7fed901522f0 legacy=0x7fed900ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed8f7fe700 1 -- 172.21.15.42:0/530009760 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290288229 0 0) 0x7fed9006ecf0 con 0x7fed900c2e90 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed9528e700 1 -- 172.21.15.42:0/530009760 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9006efd0 con 0x7fed900c92d0 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed9528e700 1 --2- 172.21.15.42:0/530009760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 0x7fed90163a10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed88009dc0 tx=0x7fed8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88652a93320aa3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.687+0000 7fed8f7fe700 1 -- 172.21.15.42:0/530009760 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800e070 con 0x7fed900c92d0 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed8f7fe700 1 -- 172.21.15.42:0/530009760 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8800bbd0 con 0x7fed900c92d0 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed8f7fe700 1 -- 172.21.15.42:0/530009760 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88012430 con 0x7fed900c92d0 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 -- 172.21.15.42:0/530009760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 msgr2=0x7fed90163a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 --2- 172.21.15.42:0/530009760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 0x7fed90163a10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fed88009dc0 tx=0x7fed8800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 -- 172.21.15.42:0/530009760 shutdown_connections 2024-05-08T23:31:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 --2- 172.21.15.42:0/530009760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed900c92d0 0x7fed90163a10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 -- 172.21.15.42:0/530009760 >> 172.21.15.42:0/530009760 conn(0x7fed90066ae0 msgr2=0x7fed90066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 -- 172.21.15.42:0/530009760 shutdown_connections 2024-05-08T23:31:54.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.688+0000 7fed96cf1700 1 -- 172.21.15.42:0/530009760 wait complete. 2024-05-08T23:31:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.689+0000 7fed96cf1700 1 Processor -- start 2024-05-08T23:31:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 -- start start 2024-05-08T23:31:54.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 0x7fed901f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed900c92d0 0x7fed90147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed90148520 con 0x7fed900c92d0 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed900cad10 con 0x7fed900c2e90 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.690+0000 7fed96cf1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed901f1870 con 0x7fed901522f0 2024-05-08T23:31:54.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed94a8d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 0x7fed901f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44260/0 (socket says 172.21.15.42:44260) 2024-05-08T23:31:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 -- 172.21.15.42:0/295390447 learned_addr learned my addr 172.21.15.42:0/295390447 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed8ffff700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed900c92d0 0x7fed90147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 -- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 msgr2=0x7fed901f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 0x7fed901f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 -- 172.21.15.42:0/295390447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed900c92d0 msgr2=0x7fed90147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed900c92d0 0x7fed90147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 -- 172.21.15.42:0/295390447 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed88009a70 con 0x7fed901522f0 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed94a8d700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 0x7fed901f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:54.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed9528e700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed9006fe30 tx=0x7fed88004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8801b810 con 0x7fed901522f0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed88020070 con 0x7fed901522f0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8801bbd0 con 0x7fed901522f0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9014b300 con 0x7fed901522f0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.691+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed900ca9a0 con 0x7fed901522f0 2024-05-08T23:31:54.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.692+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fed88027780 con 0x7fed901522f0 2024-05-08T23:31:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.692+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fed88022070 con 0x7fed901522f0 2024-05-08T23:31:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.692+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed900708c0 con 0x7fed901522f0 2024-05-08T23:31:54.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.695+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fed8802c060 con 0x7fed901522f0 2024-05-08T23:31:54.741 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:54.742+0000 7f9cfacc2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:31:54.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.786+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e7c0c6d9-0fb2-44c8-90eb-59428c93016e", "id": 6} v 0) v1 -- 0x7fed9006dfe0 con 0x7fed901522f0 2024-05-08T23:31:54.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.794+0000 7fed8d7fa700 1 -- 172.21.15.42:0/295390447 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7c0c6d9-0fb2-44c8-90eb-59428c93016e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed88035020 con 0x7fed901522f0 2024-05-08T23:31:54.793 INFO:teuthology.orchestra.run.smithi042.stdout:6 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 msgr2=0x7fed901445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed9006fe30 tx=0x7fed88004ad0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 shutdown_connections 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fed901522f0 0x7fed901445a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed900c2e90 0x7fed901f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 --2- 172.21.15.42:0/295390447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed900c92d0 0x7fed90147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 >> 172.21.15.42:0/295390447 conn(0x7fed90066ae0 msgr2=0x7fed900cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 shutdown_connections 2024-05-08T23:31:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.795+0000 7fed96cf1700 1 -- 172.21.15.42:0/295390447 wait complete. 2024-05-08T23:31:54.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new cf7a652a-2319-4560-b345-22bceefea886 7 2024-05-08T23:31:54.820 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:54.821+0000 7f8e9bcbd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:31:54.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.943+0000 7fb35c1f1700 1 Processor -- start 2024-05-08T23:31:54.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.944+0000 7fb35c1f1700 1 -- start start 2024-05-08T23:31:54.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35c1f1700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb354154970 0x7fb35406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35c1f1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35c1f1700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35415b350 con 0x7fb35406b280 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35c1f1700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35415e8a0 con 0x7fb354154970 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35c1f1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415a5d0 con 0x7fb3540caad0 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb359f8d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb354154970 0x7fb35406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35a78e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35a78e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44270/0 (socket says 172.21.15.42:44270) 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb359f8d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb354154970 0x7fb35406ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60642/0 (socket says 172.21.15.42:60642) 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb35a78e700 1 -- 172.21.15.42:0/2401167473 learned_addr learned my addr 172.21.15.42:0/2401167473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.945+0000 7fb359f8d700 1 -- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 msgr2=0x7fb35415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb359f8d700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb359f8d700 1 -- 172.21.15.42:0/2401167473 >> v1:172.21.15.42:6789/0 conn(0x7fb35406b280 legacy=0x7fb3540cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb359f8d700 1 -- 172.21.15.42:0/2401167473 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35415e5c0 con 0x7fb354154970 2024-05-08T23:31:54.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb358f8b700 1 -- 172.21.15.42:0/2401167473 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328111920 0 0) 0x7fb35415b350 con 0x7fb35406b280 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb35a78e700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb359f8d700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb354154970 0x7fb35406ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb350009d90 tx=0x7fb35000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b85eae5fc27820 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb358f8b700 1 -- 172.21.15.42:0/2401167473 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000e070 con 0x7fb354154970 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb358f8b700 1 -- 172.21.15.42:0/2401167473 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35000bb20 con 0x7fb354154970 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.946+0000 7fb358f8b700 1 -- 172.21.15.42:0/2401167473 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350012430 con 0x7fb354154970 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 -- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb354154970 msgr2=0x7fb35406ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb354154970 0x7fb35406ad40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb350009d90 tx=0x7fb35000b330 comp rx=0 tx=0).stop 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 -- 172.21.15.42:0/2401167473 shutdown_connections 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb3540caad0 0x7fb35415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/2401167473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb354154970 0x7fb35406ad40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 -- 172.21.15.42:0/2401167473 >> 172.21.15.42:0/2401167473 conn(0x7fb35406cd80 msgr2=0x7fb35406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 -- 172.21.15.42:0/2401167473 shutdown_connections 2024-05-08T23:31:54.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.947+0000 7fb35c1f1700 1 -- 172.21.15.42:0/2401167473 wait complete. 2024-05-08T23:31:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.948+0000 7fb35c1f1700 1 Processor -- start 2024-05-08T23:31:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.948+0000 7fb35c1f1700 1 -- start start 2024-05-08T23:31:54.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 0x7fb354150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 0x7fb3541f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540c5ce0 con 0x7fb354154970 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541f0230 con 0x7fb3540caad0 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35c1f1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540c6590 con 0x7fb35406b280 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb359f8d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 0x7fb354150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60644/0 (socket says 172.21.15.42:60644) 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb359f8d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 0x7fb354150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.42:44284/0 (socket says 172.21.15.42:44284) 2024-05-08T23:31:54.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 -- 172.21.15.42:0/4090269511 learned_addr learned my addr 172.21.15.42:0/4090269511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35a78e700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 0x7fb3541f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 -- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 msgr2=0x7fb354150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 0x7fb354150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 -- 172.21.15.42:0/4090269511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 msgr2=0x7fb3541f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 0x7fb3541f0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 -- 172.21.15.42:0/4090269511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c009550 con 0x7fb3540caad0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35a78e700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 0x7fb3541f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.949+0000 7fb35978c700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c520 tx=0x7fb34400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3440054e0 con 0x7fb3540caad0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb344003680 con 0x7fb3540caad0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb350009a70 con 0x7fb3540caad0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344003a30 con 0x7fb3540caad0 2024-05-08T23:31:54.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3541f41b0 con 0x7fb3540caad0 2024-05-08T23:31:54.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb34401c440 con 0x7fb3540caad0 2024-05-08T23:31:54.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.950+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb344012a60 con 0x7fb3540caad0 2024-05-08T23:31:54.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.951+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3541f03a0 con 0x7fb3540caad0 2024-05-08T23:31:54.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:54.953+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fb3541f03a0 con 0x7fb3540caad0 2024-05-08T23:31:55.000 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:55.000+0000 7f8e9bcbd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:31:55.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.045+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cf7a652a-2319-4560-b345-22bceefea886", "id": 7} v 0) v1 -- 0x7fb3541f3fe0 con 0x7fb3540caad0 2024-05-08T23:31:55.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.053+0000 7fb34a7fc700 1 -- 172.21.15.42:0/4090269511 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf7a652a-2319-4560-b345-22bceefea886", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb344017070 con 0x7fb3540caad0 2024-05-08T23:31:55.053 INFO:teuthology.orchestra.run.smithi042.stdout:7 2024-05-08T23:31:55.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 msgr2=0x7fb3541f0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c520 tx=0x7fb34400c830 comp rx=0 tx=0).stop 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 shutdown_connections 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb354154970 0x7fb3541f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb35406b280 0x7fb354150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 --2- 172.21.15.42:0/4090269511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3540caad0 0x7fb3541f0680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 >> 172.21.15.42:0/4090269511 conn(0x7fb35406cd80 msgr2=0x7fb3541531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 shutdown_connections 2024-05-08T23:31:55.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-05-08T23:31:55.054+0000 7fb35c1f1700 1 -- 172.21.15.42:0/4090269511 wait complete. 2024-05-08T23:31:55.068 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T23:31:55.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T23:31:55.070 INFO:tasks.ceph.osd.0:Started 2024-05-08T23:31:55.071 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T23:31:55.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T23:31:55.075 INFO:tasks.ceph.osd.1:Started 2024-05-08T23:31:55.075 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T23:31:55.075 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T23:31:55.077 INFO:tasks.ceph.osd.2:Started 2024-05-08T23:31:55.077 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T23:31:55.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T23:31:55.079 INFO:tasks.ceph.osd.3:Started 2024-05-08T23:31:55.080 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T23:31:55.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T23:31:55.082 INFO:tasks.ceph.osd.4:Started 2024-05-08T23:31:55.082 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T23:31:55.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T23:31:55.085 INFO:tasks.ceph.osd.5:Started 2024-05-08T23:31:55.085 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T23:31:55.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T23:31:55.087 INFO:tasks.ceph.osd.6:Started 2024-05-08T23:31:55.087 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T23:31:55.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T23:31:55.091 INFO:tasks.ceph.osd.7:Started 2024-05-08T23:31:55.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:55.095 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:55.095+0000 7f8e9bcbd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:31:55.130 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:55.130+0000 7f9cfacc2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:31:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.267+0000 7f514291d700 1 Processor -- start 2024-05-08T23:31:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.267+0000 7f514291d700 1 -- start start 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.268+0000 7f514291d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f513c0c93d0 0x7f513c0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.268+0000 7f514291d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c1581a0 con 0x7f513c15b680 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.268+0000 7f514291d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0c97b0 con 0x7f513c0666c0 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.268+0000 7f514291d700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c158410 con 0x7f513c0c93d0 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f5140eba700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f513c15b680 0x7f513c15ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41692/0 (socket says 172.21.15.33:41692) 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f5140eba700 1 -- 172.21.15.33:0/3398786774 learned_addr learned my addr 172.21.15.33:0/3398786774 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513bfff700 1 --2- 172.21.15.33:0/3398786774 >> v2:172.21.15.42:3300/0 conn(0x7f513c0c93d0 0x7f513c0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513affd700 1 -- 172.21.15.33:0/3398786774 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269944200 0 0) 0x7f513c1581a0 con 0x7f513c15b680 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513affd700 1 -- 172.21.15.33:0/3398786774 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51240035b0 con 0x7f513c15b680 2024-05-08T23:31:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513bfff700 1 -- 172.21.15.33:0/3398786774 >> v1:172.21.15.42:6789/0 conn(0x7f513c0666c0 legacy=0x7f513c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513bfff700 1 -- 172.21.15.33:0/3398786774 >> v1:172.21.15.33:6789/0 conn(0x7f513c15b680 legacy=0x7f513c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513bfff700 1 -- 172.21.15.33:0/3398786774 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c1586f0 con 0x7f513c0c93d0 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513bfff700 1 --2- 172.21.15.33:0/3398786774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c0c93d0 0x7f513c0c6f90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f512c009840 tx=0x7f512c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=323a69cce9073314 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513affd700 1 -- 172.21.15.33:0/3398786774 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c011830 con 0x7f513c0c93d0 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513affd700 1 -- 172.21.15.33:0/3398786774 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c011990 con 0x7f513c0c93d0 2024-05-08T23:31:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.269+0000 7f513affd700 1 -- 172.21.15.33:0/3398786774 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c011d20 con 0x7f513c0c93d0 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 -- 172.21.15.33:0/3398786774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c0c93d0 msgr2=0x7f513c0c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 --2- 172.21.15.33:0/3398786774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c0c93d0 0x7f513c0c6f90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f512c009840 tx=0x7f512c009b50 comp rx=0 tx=0).stop 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 -- 172.21.15.33:0/3398786774 shutdown_connections 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 --2- 172.21.15.33:0/3398786774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c0c93d0 0x7f513c0c6f90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 -- 172.21.15.33:0/3398786774 >> 172.21.15.33:0/3398786774 conn(0x7f513c06a990 msgr2=0x7f513c06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 -- 172.21.15.33:0/3398786774 shutdown_connections 2024-05-08T23:31:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.270+0000 7f514291d700 1 -- 172.21.15.33:0/3398786774 wait complete. 2024-05-08T23:31:55.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.271+0000 7f514291d700 1 Processor -- start 2024-05-08T23:31:55.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 -- start start 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f513c0c93d0 0x7f513c072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c15b680 0x7f513c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1f28e0 con 0x7f513c0666c0 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1f0460 con 0x7f513c15b680 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.272+0000 7f514291d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1eff30 con 0x7f513c0c93d0 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60882/0 (socket says 172.21.15.33:60882) 2024-05-08T23:31:55.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 -- 172.21.15.33:0/2375992549 learned_addr learned my addr 172.21.15.33:0/2375992549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f5140eba700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c15b680 0x7f513c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513b7fe700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f513c0c93d0 0x7f513c072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 -- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f513c0c93d0 msgr2=0x7f513c072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f513c0c93d0 0x7f513c072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 -- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c15b680 msgr2=0x7f513c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c15b680 0x7f513c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 -- 172.21.15.33:0/2375992549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0094a0 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f513bfff700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f513c072a50 tx=0x7f512c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0036a0 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c01f070 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c148720 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c003a60 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.273+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1f0180 con 0x7f513c0666c0 2024-05-08T23:31:55.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.274+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f512c026480 con 0x7f513c0666c0 2024-05-08T23:31:55.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.274+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f512c021070 con 0x7f513c0666c0 2024-05-08T23:31:55.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.275+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c075460 con 0x7f513c0666c0 2024-05-08T23:31:55.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.277+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f512c015020 con 0x7f513c0666c0 2024-05-08T23:31:55.327 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:55.328+0000 7f9cfacc2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:31:55.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.352+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f513c14bb80 con 0x7f513c0666c0 2024-05-08T23:31:55.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.353+0000 7f5138ff9700 1 -- 172.21.15.33:0/2375992549 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f512c026920 con 0x7f513c0666c0 2024-05-08T23:31:55.353 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:31:55.353 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:31:55.047509+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 msgr2=0x7f513c1f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f513c072a50 tx=0x7f512c004750 comp rx=0 tx=0).stop 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 shutdown_connections 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f513c15b680 0x7f513c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f513c0666c0 0x7f513c1f23a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 --2- 172.21.15.33:0/2375992549 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f513c0c93d0 0x7f513c072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:55.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 >> 172.21.15.33:0/2375992549 conn(0x7f513c06a990 msgr2=0x7f513c0c59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:55.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 shutdown_connections 2024-05-08T23:31:55.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:31:55.354+0000 7f514291d700 1 -- 172.21.15.33:0/2375992549 wait complete. 2024-05-08T23:31:55.368 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T23:31:55.368 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T23:31:55.416 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:55.417+0000 7f9cfacc2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:31:55.450 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-05-08T23:31:55.450+0000 7fb0e561b7c0 -1 Falling back to public interface 2024-05-08T23:31:55.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-05-08T23:31:55.702+0000 7f8182bfa7c0 -1 Falling back to public interface 2024-05-08T23:31:55.706 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-05-08T23:31:55.706+0000 7f31e79a57c0 -1 Falling back to public interface 2024-05-08T23:31:55.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-05-08T23:31:55.717+0000 7fa0ba1207c0 -1 Falling back to public interface 2024-05-08T23:31:55.718 INFO:tasks.ceph.osd.7.smithi042.stderr:2024-05-08T23:31:55.718+0000 7efdd31177c0 -1 Falling back to public interface 2024-05-08T23:31:55.731 INFO:tasks.ceph.osd.6.smithi042.stderr:2024-05-08T23:31:55.731+0000 7f3e16aaf7c0 -1 Falling back to public interface 2024-05-08T23:31:55.734 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-05-08T23:31:55.735+0000 7f3a5ed9c7c0 -1 Falling back to public interface 2024-05-08T23:31:55.738 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-05-08T23:31:55.738+0000 7f8e2d1337c0 -1 Falling back to public interface 2024-05-08T23:31:55.956 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:55.956+0000 7f8e9bcbd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:31:56.050 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:56.050+0000 7f8e9bcbd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:31:56.177 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:56.202 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:56.203+0000 7f8e9bcbd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:31:56.219 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:56.220+0000 7f9cfacc2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:31:56.303 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.304+0000 7f6637990700 1 Processor -- start 2024-05-08T23:31:56.304 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.304+0000 7f6637990700 1 -- start start 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66300b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f66300cc970 0x7f66300cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663006ab40 0x7f663006af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66300c7860 con 0x7f663006ab40 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66300c8100 con 0x7f66300cc970 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6637990700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300cb1a0 con 0x7f66300cc590 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6635f2d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663006ab40 0x7f663006af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6635f2d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f663006ab40 0x7f663006af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44024/0 (socket says 172.21.15.33:44024) 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6635f2d700 1 -- 172.21.15.33:0/1597796107 learned_addr learned my addr 172.21.15.33:0/1597796107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f663572c700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66300b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.307 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.305+0000 7f6634f2b700 1 --2- 172.21.15.33:0/1597796107 >> v2:172.21.15.42:3300/0 conn(0x7f66300cc970 0x7f66300cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 -- 172.21.15.33:0/1597796107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 msgr2=0x7f66300b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66300b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 -- 172.21.15.33:0/1597796107 >> v2:172.21.15.42:3300/0 conn(0x7f66300cc970 msgr2=0x7f66300cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 --2- 172.21.15.33:0/1597796107 >> v2:172.21.15.42:3300/0 conn(0x7f66300cc970 0x7f66300cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 -- 172.21.15.33:0/1597796107 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300700e0 con 0x7f663006ab40 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6635f2d700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663006ab40 0x7f663006af90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66300c40e0 tx=0x7f662800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc27e727eed5ab77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6627fff700 1 -- 172.21.15.33:0/1597796107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662800e070 con 0x7f663006ab40 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6627fff700 1 -- 172.21.15.33:0/1597796107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662800b910 con 0x7f663006ab40 2024-05-08T23:31:56.308 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.306+0000 7f6627fff700 1 -- 172.21.15.33:0/1597796107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662800bcc0 con 0x7f663006ab40 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 -- 172.21.15.33:0/1597796107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663006ab40 msgr2=0x7f663006af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663006ab40 0x7f663006af90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66300c40e0 tx=0x7f662800b330 comp rx=0 tx=0).stop 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 -- 172.21.15.33:0/1597796107 shutdown_connections 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f663006ab40 0x7f663006af90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 --2- 172.21.15.33:0/1597796107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66300b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 --2- 172.21.15.33:0/1597796107 >> v2:172.21.15.42:3300/0 conn(0x7f66300cc970 0x7f66300cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 -- 172.21.15.33:0/1597796107 >> 172.21.15.33:0/1597796107 conn(0x7f66300cce60 msgr2=0x7f66300cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 -- 172.21.15.33:0/1597796107 shutdown_connections 2024-05-08T23:31:56.309 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.307+0000 7f6637990700 1 -- 172.21.15.33:0/1597796107 wait complete. 2024-05-08T23:31:56.313 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.309+0000 7f6637990700 1 Processor -- start 2024-05-08T23:31:56.313 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.309+0000 7f6637990700 1 -- start start 2024-05-08T23:31:56.313 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 0x7f663014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66301efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301536b0 con 0x7f66300cc970 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301f4210 con 0x7f663006ab40 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6637990700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66301f3d80 con 0x7f66300cc590 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f663572c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 0x7f663014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.314 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6634f2b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66301efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44026/0 (socket says 172.21.15.33:44026) 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f663572c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 0x7f663014ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:40972/0 (socket says 172.21.15.33:40972) 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 -- 172.21.15.33:0/4219566618 learned_addr learned my addr 172.21.15.33:0/4219566618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 -- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 msgr2=0x7f66301efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66301efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 -- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 msgr2=0x7f663014ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 0x7f663014ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.315 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 -- 172.21.15.33:0/4219566618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628009b30 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6634f2b700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66301efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6635f2d700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f662800b680 tx=0x7f6628004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.310+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280092e0 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662800ce80 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280044b0 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66301533d0 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66301f3730 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f662801b430 con 0x7f66300cc970 2024-05-08T23:31:56.316 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.311+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6628024ae0 con 0x7f66300cc970 2024-05-08T23:31:56.317 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.312+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66301f3ef0 con 0x7f66300cc970 2024-05-08T23:31:56.317 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:56.315+0000 7f9cfacc2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:31:56.318 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.316+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f662801f030 con 0x7f66300cc970 2024-05-08T23:31:56.392 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.393+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66300753d0 con 0x7f66300cc970 2024-05-08T23:31:56.393 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.394+0000 7f6625ffb700 1 -- 172.21.15.33:0/4219566618 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f662801b970 con 0x7f66300cc970 2024-05-08T23:31:56.393 INFO:teuthology.misc.health.smithi033.stdout: 2024-05-08T23:31:56.393 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:31:55.047509+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 msgr2=0x7f66301f0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f662800b680 tx=0x7f6628004ad0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 shutdown_connections 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66300cc970 0x7f66301f0280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f663006ab40 0x7f663014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 --2- 172.21.15.33:0/4219566618 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66300cc590 0x7f66301efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 >> 172.21.15.33:0/4219566618 conn(0x7f66300cce60 msgr2=0x7f66300c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 shutdown_connections 2024-05-08T23:31:56.395 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:31:56.395+0000 7f6637990700 1 -- 172.21.15.33:0/4219566618 wait complete. 2024-05-08T23:31:56.408 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T23:31:56.465 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:56.465+0000 7f9cfacc2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:31:56.847 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:56.847+0000 7f8e9bcbd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:31:57.023 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:57.023+0000 7f8e9bcbd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:31:57.098 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:57.098+0000 7f9cfacc2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:31:57.210 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:57.211+0000 7f8e9bcbd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:31:57.285 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:57.286+0000 7f9cfacc2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:31:57.474 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:57.474+0000 7f9cfacc2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:31:58.142 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:58.143+0000 7f8e9bcbd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:31:58.423 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:58.424+0000 7f9cfacc2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:31:58.424 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:58.425+0000 7f8e9bcbd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:31:58.549 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:58.549+0000 7f8e9bcbd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:31:58.744 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:58.745+0000 7f9cfacc2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:31:58.748 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:58.748+0000 7f8e9bcbd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:31:58.834 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:58.832+0000 7f9cfacc2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:31:58.972 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-05-08T23:31:58.972+0000 7fb0e561b7c0 -1 osd.5 0 log_to_monitors true 2024-05-08T23:31:59.027 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-05-08T23:31:59.028+0000 7f9cfacc2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:31:59.209 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-05-08T23:31:59.209+0000 7f31e79a57c0 -1 osd.2 0 log_to_monitors true 2024-05-08T23:31:59.224 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-05-08T23:31:59.225+0000 7fa0ba1207c0 -1 osd.1 0 log_to_monitors true 2024-05-08T23:31:59.229 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-05-08T23:31:59.230+0000 7f8182bfa7c0 -1 osd.0 0 log_to_monitors true 2024-05-08T23:31:59.235 INFO:tasks.ceph.osd.6.smithi042.stderr:2024-05-08T23:31:59.236+0000 7f3e16aaf7c0 -1 osd.6 0 log_to_monitors true 2024-05-08T23:31:59.245 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-05-08T23:31:59.245+0000 7f3a5ed9c7c0 -1 osd.4 0 log_to_monitors true 2024-05-08T23:31:59.252 INFO:tasks.ceph.osd.7.smithi042.stderr:2024-05-08T23:31:59.253+0000 7efdd31177c0 -1 osd.7 0 log_to_monitors true 2024-05-08T23:31:59.262 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-05-08T23:31:59.263+0000 7f8e2d1337c0 -1 osd.3 0 log_to_monitors true 2024-05-08T23:31:59.774 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:59.775+0000 7f8e56f59700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T23:31:59.775 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-05-08T23:31:59.775+0000 7f8e56f59700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T23:32:02.087 INFO:tasks.ceph.osd.6.smithi042.stderr:2024-05-08T23:32:02.088+0000 7f3e0a664700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T23:32:02.102 INFO:tasks.ceph.osd.6.smithi042.stderr:2024-05-08T23:32:02.103+0000 7f3e02a3f700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.174 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-05-08T23:32:02.175+0000 7f3a52951700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T23:32:02.188 INFO:tasks.ceph.osd.4.smithi042.stderr:2024-05-08T23:32:02.189+0000 7f3a4ad2c700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.189 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-05-08T23:32:02.189+0000 7fb0d91d0700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T23:32:02.203 INFO:tasks.ceph.osd.5.smithi042.stderr:2024-05-08T23:32:02.204+0000 7fb0d15ab700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.246 INFO:tasks.ceph.osd.7.smithi042.stderr:2024-05-08T23:32:02.247+0000 7efdc6ccc700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T23:32:02.264 INFO:tasks.ceph.osd.7.smithi042.stderr:2024-05-08T23:32:02.265+0000 7efdbf0a7700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.858 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-05-08T23:32:02.859+0000 7fa0adcd5700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T23:32:02.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-05-08T23:32:02.874+0000 7f81767af700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T23:32:02.878 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-05-08T23:32:02.879+0000 7fa0a60b0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.891 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-05-08T23:32:02.892+0000 7f816eb8a700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.920 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-05-08T23:32:02.921+0000 7f31db55a700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T23:32:02.930 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-05-08T23:32:02.929+0000 7f8e20ce8700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T23:32:02.931 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-05-08T23:32:02.932+0000 7f31d3935700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:02.948 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-05-08T23:32:02.949+0000 7f8e190c3700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:32:03.213 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:03.347 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.347+0000 7f87f3be6700 1 Processor -- start 2024-05-08T23:32:03.347 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.347+0000 7f87f3be6700 1 -- start start 2024-05-08T23:32:03.349 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f3be6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87ec070b40 0x7f87ec0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.349 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f3be6700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 0x7f87ec158db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f3be6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec15bc90 con 0x7f87ec070760 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f3be6700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec1592f0 con 0x7f87ec070b40 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f3be6700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec15bf30 con 0x7f87ec158990 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f1181700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87ec070b40 0x7f87ec0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f1181700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87ec070b40 0x7f87ec0c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44130/0 (socket says 172.21.15.33:44130) 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f1982700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f87ec070760 0x7f87ec0cb020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53766/0 (socket says 172.21.15.33:53766) 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.348+0000 7f87f1181700 1 -- 172.21.15.33:0/1392556863 learned_addr learned my addr 172.21.15.33:0/1392556863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f2183700 1 --2- 172.21.15.33:0/1392556863 >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 0x7f87ec158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f1181700 1 -- 172.21.15.33:0/1392556863 >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 msgr2=0x7f87ec158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f1181700 1 --2- 172.21.15.33:0/1392556863 >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 0x7f87ec158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f1181700 1 -- 172.21.15.33:0/1392556863 >> v1:172.21.15.33:6789/0 conn(0x7f87ec070760 legacy=0x7f87ec0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.350 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f1181700 1 -- 172.21.15.33:0/1392556863 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec15c210 con 0x7f87ec070b40 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f0980700 1 -- 172.21.15.33:0/1392556863 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562069233 0 0) 0x7f87ec15bc90 con 0x7f87ec070760 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f2183700 1 --2- 172.21.15.33:0/1392556863 >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 0x7f87ec158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f1181700 1 --2- 172.21.15.33:0/1392556863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec070b40 0x7f87ec0c4a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0c8260 tx=0x7f87dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb5e079529c755 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f0980700 1 -- 172.21.15.33:0/1392556863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00b870 con 0x7f87ec070b40 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f0980700 1 -- 172.21.15.33:0/1392556863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc00b9d0 con 0x7f87ec070b40 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.349+0000 7f87f0980700 1 -- 172.21.15.33:0/1392556863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00bd60 con 0x7f87ec070b40 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 -- 172.21.15.33:0/1392556863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec070b40 msgr2=0x7f87ec0c4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 --2- 172.21.15.33:0/1392556863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec070b40 0x7f87ec0c4a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0c8260 tx=0x7f87dc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 -- 172.21.15.33:0/1392556863 shutdown_connections 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 --2- 172.21.15.33:0/1392556863 >> v2:172.21.15.42:3300/0 conn(0x7f87ec158990 0x7f87ec158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 --2- 172.21.15.33:0/1392556863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec070b40 0x7f87ec0c4a70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 -- 172.21.15.33:0/1392556863 >> 172.21.15.33:0/1392556863 conn(0x7f87ec0731e0 msgr2=0x7f87ec06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 -- 172.21.15.33:0/1392556863 shutdown_connections 2024-05-08T23:32:03.351 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.350+0000 7f87f3be6700 1 -- 172.21.15.33:0/1392556863 wait complete. 2024-05-08T23:32:03.354 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.351+0000 7f87f3be6700 1 Processor -- start 2024-05-08T23:32:03.354 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.351+0000 7f87f3be6700 1 -- start start 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 0x7f87ec1efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec070b40 0x7f87ec1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1f1100 con 0x7f87ec158990 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1f0be0 con 0x7f87ec070b40 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f3be6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1f0340 con 0x7f87ec070760 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44132/0 (socket says 172.21.15.33:44132) 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 -- 172.21.15.33:0/3357687829 learned_addr learned my addr 172.21.15.33:0/3357687829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f1181700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec070b40 0x7f87ec1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f1982700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 0x7f87ec1efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 -- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 msgr2=0x7f87ec1efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 0x7f87ec1efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 -- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec070b40 msgr2=0x7f87ec1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.355 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec070b40 0x7f87ec1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f2183700 1 -- 172.21.15.33:0/3357687829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009550 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.352+0000 7f87f1982700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 0x7f87ec1efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87f2183700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87e400cd50 tx=0x7f87e400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400cea0 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e400f480 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1f0620 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400f810 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.353+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec0cd0e0 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.354+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0c9790 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.354+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f87e400b040 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.355+0000 7f87e27fc700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 0x7f87d006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.355+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f87e4089500 con 0x7f87ec158990 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.355+0000 7f87f1982700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 0x7f87d006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.356 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.355+0000 7f87f1982700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 0x7f87d006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.359 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.360+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f87e4054c90 con 0x7f87ec158990 2024-05-08T23:32:03.577 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.577+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87ec0753d0 con 0x7f87ec158990 2024-05-08T23:32:03.577 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.578+0000 7f87e27fc700 1 -- 172.21.15.33:0/3357687829 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10392 (secure 0 0 0) 0x7f87e401d090 con 0x7f87ec158990 2024-05-08T23:32:03.578 INFO:teuthology.misc.health.smithi033.stdout: 2024-05-08T23:32:03.578 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":13,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:03.174271+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:03.580 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.580+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 msgr2=0x7f87d006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.580 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.580+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 0x7f87d006d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8007ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.580 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.580+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 msgr2=0x7f87ec0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.580 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.580+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87e400cd50 tx=0x7f87e400a560 comp rx=0 tx=0).stop 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 shutdown_connections 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87d006ac00 0x7f87d006d0c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec158990 0x7f87ec0cd8e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87ec070760 0x7f87ec1efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 --2- 172.21.15.33:0/3357687829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec070b40 0x7f87ec1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 >> 172.21.15.33:0/3357687829 conn(0x7f87ec0731e0 msgr2=0x7f87ec0c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 shutdown_connections 2024-05-08T23:32:03.581 INFO:teuthology.misc.health.smithi033.stderr:2024-05-08T23:32:03.581+0000 7f87f3be6700 1 -- 172.21.15.33:0/3357687829 wait complete. 2024-05-08T23:32:03.593 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T23:32:03.593 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T23:32:03.593 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T23:32:03.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T23:32:03.596 INFO:tasks.ceph.mds.a:Started 2024-05-08T23:32:03.596 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T23:32:03.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T23:32:03.598 INFO:tasks.ceph.mds.c:Started 2024-05-08T23:32:03.598 INFO:tasks.ceph.mds.e:Restarting daemon 2024-05-08T23:32:03.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-05-08T23:32:03.600 INFO:tasks.ceph.mds.e:Started 2024-05-08T23:32:03.600 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T23:32:03.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T23:32:03.604 INFO:tasks.ceph.mds.b:Started 2024-05-08T23:32:03.605 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T23:32:03.605 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T23:32:03.608 INFO:tasks.ceph.mds.d:Started 2024-05-08T23:32:03.608 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-08T23:32:03.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-08T23:32:03.611 INFO:tasks.ceph.mds.f:Started 2024-05-08T23:32:03.611 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T23:32:03.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:03.690 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2024-05-08T23:32:03.691 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2024-05-08T23:32:03.699 INFO:tasks.ceph.mds.e.smithi033.stdout:starting mds.e at 2024-05-08T23:32:03.704 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-05-08T23:32:03.708 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-05-08T23:32:03.708 INFO:tasks.ceph.mds.f.smithi042.stdout:starting mds.f at 2024-05-08T23:32:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.760+0000 7f77ab2f5700 1 Processor -- start 2024-05-08T23:32:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.761+0000 7f77ab2f5700 1 -- start start 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.761+0000 7f77ab2f5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f77a40c68e0 0x7f77a4071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.761+0000 7f77ab2f5700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a41582e0 con 0x7f77a40c6e30 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.761+0000 7f77ab2f5700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a4157e50 con 0x7f77a40c6310 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.761+0000 7f77ab2f5700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a4157310 con 0x7f77a40c68e0 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a9892700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f77a40c6e30 0x7f77a4161bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53790/0 (socket says 172.21.15.33:53790) 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a9892700 1 -- 172.21.15.33:0/268255415 learned_addr learned my addr 172.21.15.33:0/268255415 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a8890700 1 --2- 172.21.15.33:0/268255415 >> v2:172.21.15.42:3300/0 conn(0x7f77a40c68e0 0x7f77a4071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922871691 0 0) 0x7f77a41582e0 con 0x7f77a40c6e30 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77900035b0 con 0x7f77a40c6e30 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551748960 0 0) 0x7f77a4157e50 con 0x7f77a40c6310 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a41582e0 con 0x7f77a40c6310 2024-05-08T23:32:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a8890700 1 -- 172.21.15.33:0/268255415 >> v1:172.21.15.42:6789/0 conn(0x7f77a40c6310 legacy=0x7f77a406baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a8890700 1 -- 172.21.15.33:0/268255415 >> v1:172.21.15.33:6789/0 conn(0x7f77a40c6e30 legacy=0x7f77a4161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a8890700 1 -- 172.21.15.33:0/268255415 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40c66f0 con 0x7f77a40c68e0 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f77a8890700 1 --2- 172.21.15.33:0/268255415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c68e0 0x7f77a4071710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7794009840 tx=0x7f7794009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf32f29456fa5e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.762+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794011830 con 0x7f77a40c68e0 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794011990 con 0x7f77a40c68e0 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f779bfff700 1 -- 172.21.15.33:0/268255415 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794011d20 con 0x7f77a40c68e0 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 -- 172.21.15.33:0/268255415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c68e0 msgr2=0x7f77a4071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/268255415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c68e0 0x7f77a4071710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7794009840 tx=0x7f7794009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 -- 172.21.15.33:0/268255415 shutdown_connections 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/268255415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c68e0 0x7f77a4071710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 -- 172.21.15.33:0/268255415 >> 172.21.15.33:0/268255415 conn(0x7f77a406d2b0 msgr2=0x7f77a406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 -- 172.21.15.33:0/268255415 shutdown_connections 2024-05-08T23:32:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.763+0000 7f77ab2f5700 1 -- 172.21.15.33:0/268255415 wait complete. 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.764+0000 7f77ab2f5700 1 Processor -- start 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 -- start start 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c6310 0x7f77a40c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77a40c6e30 0x7f77a4149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4153d20 con 0x7f77a40c68e0 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40c08c0 con 0x7f77a40c6310 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77ab2f5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41f3f10 con 0x7f77a40c6e30 2024-05-08T23:32:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77a8890700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77a8890700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44202/0 (socket says 172.21.15.33:44202) 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.765+0000 7f77a8890700 1 -- 172.21.15.33:0/3657514354 learned_addr learned my addr 172.21.15.33:0/3657514354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a9892700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77a40c6e30 0x7f77a4149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a9091700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c6310 0x7f77a40c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 -- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77a40c6e30 msgr2=0x7f77a4149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77a40c6e30 0x7f77a4149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 -- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c6310 msgr2=0x7f77a40c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c6310 0x7f77a40c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 -- 172.21.15.33:0/3657514354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940094a0 con 0x7f77a40c68e0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77a8890700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77940090b0 tx=0x7f7794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779401a860 con 0x7f77a40c68e0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779401a9c0 con 0x7f77a40c68e0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a41f41f0 con 0x7f77a40c68e0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779401ad30 con 0x7f77a40c68e0 2024-05-08T23:32:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.766+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a4154250 con 0x7f77a40c68e0 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.767+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4075460 con 0x7f77a40c68e0 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.768+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f7794022070 con 0x7f77a40c68e0 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.768+0000 7f7799ffb700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 0x7f778806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.768+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f77940899e0 con 0x7f77a40c68e0 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.768+0000 7f77a9091700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 0x7f778806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:03.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.769+0000 7f77a9091700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 0x7f778806d0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77a0007fd0 tx=0x7f77a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.773+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f77940552b0 con 0x7f77a40c68e0 2024-05-08T23:32:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.994+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a4072d80 con 0x7f77a40c68e0 2024-05-08T23:32:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.994+0000 7f7799ffb700 1 -- 172.21.15.33:0/3657514354 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10392 (secure 0 0 0) 0x7f7794029090 con 0x7f77a40c68e0 2024-05-08T23:32:03.994 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:03.994 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:03.174271+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 msgr2=0x7f778806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 0x7f778806d0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77a0007fd0 tx=0x7f77a0009040 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 msgr2=0x7f77a41504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77940090b0 tx=0x7f7794004750 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 shutdown_connections 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f778806ac30 0x7f778806d0f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77a40c6e30 0x7f77a4149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77a40c6310 0x7f77a40c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 --2- 172.21.15.33:0/3657514354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a40c68e0 0x7f77a41504c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 >> 172.21.15.33:0/3657514354 conn(0x7f77a406d2b0 msgr2=0x7f77a4156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 shutdown_connections 2024-05-08T23:32:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:03.997+0000 7f77ab2f5700 1 -- 172.21.15.33:0/3657514354 wait complete. 2024-05-08T23:32:04.010 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-08T23:32:04.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.147+0000 7f0b962e5700 1 Processor -- start 2024-05-08T23:32:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.147+0000 7f0b962e5700 1 -- start start 2024-05-08T23:32:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b962e5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900c93d0 0x7f0b900c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b962e5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b962e5700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b901581a0 con 0x7f0b900666c0 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b962e5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900c97b0 con 0x7f0b900c93d0 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b962e5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90158410 con 0x7f0b9015b680 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900c93d0 0x7f0b900c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0b900c93d0 0x7f0b900c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44210/0 (socket says 172.21.15.33:44210) 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 -- 172.21.15.33:0/66199104 learned_addr learned my addr 172.21.15.33:0/66199104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b94882700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 -- 172.21.15.33:0/66199104 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 msgr2=0x7f0b9015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 -- 172.21.15.33:0/66199104 >> v1:172.21.15.42:6789/0 conn(0x7f0b900666c0 legacy=0x7f0b900c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b8ffff700 1 -- 172.21.15.33:0/66199104 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b901586f0 con 0x7f0b900c93d0 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.148+0000 7f0b94882700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900c93d0 0x7f0b900c6f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098f0 tx=0x7f0b80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9f860847d95a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b8effd700 1 -- 172.21.15.33:0/66199104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000b8a0 con 0x7f0b900c93d0 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b8effd700 1 -- 172.21.15.33:0/66199104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8000ba00 con 0x7f0b900c93d0 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b8effd700 1 -- 172.21.15.33:0/66199104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000bd90 con 0x7f0b900c93d0 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 -- 172.21.15.33:0/66199104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900c93d0 msgr2=0x7f0b900c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900c93d0 0x7f0b900c6f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b800098f0 tx=0x7f0b80009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 -- 172.21.15.33:0/66199104 shutdown_connections 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 --2- 172.21.15.33:0/66199104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900c93d0 0x7f0b900c6f90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 -- 172.21.15.33:0/66199104 >> 172.21.15.33:0/66199104 conn(0x7f0b9006a990 msgr2=0x7f0b9006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 -- 172.21.15.33:0/66199104 shutdown_connections 2024-05-08T23:32:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.149+0000 7f0b962e5700 1 -- 172.21.15.33:0/66199104 wait complete. 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.151+0000 7f0b962e5700 1 Processor -- start 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.151+0000 7f0b962e5700 1 -- start start 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b900c93d0 0x7f0b9015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900707e0 con 0x7f0b900666c0 2024-05-08T23:32:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90147bf0 con 0x7f0b900c93d0 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b962e5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90070a30 con 0x7f0b9015b680 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44214/0 (socket says 172.21.15.33:44214) 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 -- 172.21.15.33:0/3856304825 learned_addr learned my addr 172.21.15.33:0/3856304825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8f7fe700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b900c93d0 0x7f0b9015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b94882700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 -- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 msgr2=0x7f0b9015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 -- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b900c93d0 msgr2=0x7f0b9015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b900c93d0 0x7f0b9015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 -- 172.21.15.33:0/3856304825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80009550 con 0x7f0b900666c0 2024-05-08T23:32:04.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.152+0000 7f0b8ffff700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000c00 tx=0x7f0b80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.153+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80004300 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.153+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8001f070 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.153+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b900717a0 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.153+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800036a0 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.153+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b90071d60 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.154+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b90075460 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.154+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f0b80021030 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.154+0000 7f0b8cff9700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 0x7f0b7006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.155+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f0b8008a080 con 0x7f0b900666c0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.155+0000 7f0b8f7fe700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 0x7f0b7006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:04.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.155+0000 7f0b8f7fe700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 0x7f0b7006d0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b9014ea40 tx=0x7f0b78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:04.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.159+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0b80055820 con 0x7f0b900666c0 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.433+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b900c6d70 con 0x7f0b900666c0 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.434+0000 7f0b8cff9700 1 -- 172.21.15.33:0/3856304825 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f0b80058e40 con 0x7f0b900666c0 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 msgr2=0x7f0b7006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 0x7f0b7006d0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b9014ea40 tx=0x7f0b78009040 comp rx=0 tx=0).stop 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 msgr2=0x7f0b901476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b80000c00 tx=0x7f0b80004750 comp rx=0 tx=0).stop 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 shutdown_connections 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0b7006ac00 0x7f0b7006d0c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0b9015b680 0x7f0b9015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b900666c0 0x7f0b901476b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 --2- 172.21.15.33:0/3856304825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b900c93d0 0x7f0b9015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 >> 172.21.15.33:0/3856304825 conn(0x7f0b9006a990 msgr2=0x7f0b900c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.436+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 shutdown_connections 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:04.437+0000 7f0b962e5700 1 -- 172.21.15.33:0/3856304825 wait complete. 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-05-08T23:32:06.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.597+0000 7f011e4ae700 1 Processor -- start 2024-05-08T23:32:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.597+0000 7f011e4ae700 1 -- start start 2024-05-08T23:32:06.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.597+0000 7f011e4ae700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f011e4ae700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01180c8e50 0x7f01180ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f011e4ae700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011806baa0 con 0x7f011815dd40 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f011e4ae700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011806bcf0 con 0x7f01180c8e50 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f011e4ae700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011806bfd0 con 0x7f0118154520 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f0117fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01180c8e50 0x7f01180ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f01180c8e50 0x7f01180ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:42192/0 (socket says 172.21.15.33:42192) 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f0117fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:47820/0 (socket says 172.21.15.33:47820) 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 -- 172.21.15.33:0/1392360112 learned_addr learned my addr 172.21.15.33:0/1392360112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f011ca4b700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f011815dd40 0x7f0118070140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:52504/0 (socket says 172.21.15.33:52504) 2024-05-08T23:32:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 -- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 msgr2=0x7f01180cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 -- 172.21.15.33:0/1392360112 >> v1:172.21.15.42:6789/0 conn(0x7f011815dd40 legacy=0x7f0118070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f01177fe700 1 -- 172.21.15.33:0/1392360112 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011806c2b0 con 0x7f01180c8e50 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.598+0000 7f0117fff700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f01177fe700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01180c8e50 0x7f01180ce270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01000098f0 tx=0x7f0100009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38023f615715f36f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f0116ffd700 1 -- 172.21.15.33:0/1392360112 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100011830 con 0x7f01180c8e50 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f0116ffd700 1 -- 172.21.15.33:0/1392360112 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100011990 con 0x7f01180c8e50 2024-05-08T23:32:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f0116ffd700 1 -- 172.21.15.33:0/1392360112 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100011d20 con 0x7f01180c8e50 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f011e4ae700 1 -- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01180c8e50 msgr2=0x7f01180ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.599+0000 7f011e4ae700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01180c8e50 0x7f01180ce270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f01000098f0 tx=0x7f0100009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 -- 172.21.15.33:0/1392360112 shutdown_connections 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0118154520 0x7f01180cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 --2- 172.21.15.33:0/1392360112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01180c8e50 0x7f01180ce270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 -- 172.21.15.33:0/1392360112 >> 172.21.15.33:0/1392360112 conn(0x7f0118072230 msgr2=0x7f0118072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 -- 172.21.15.33:0/1392360112 shutdown_connections 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.600+0000 7f011e4ae700 1 -- 172.21.15.33:0/1392360112 wait complete. 2024-05-08T23:32:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.601+0000 7f011e4ae700 1 Processor -- start 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.601+0000 7f011e4ae700 1 -- start start 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01180c8e50 0x7f01181f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011815dd40 0x7f01181eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01181ee540 con 0x7f0118154520 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180c3040 con 0x7f011815dd40 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011e4ae700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011814d820 con 0x7f01180c8e50 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48332/0 (socket says 172.21.15.33:48332) 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 -- 172.21.15.33:0/54688147 learned_addr learned my addr 172.21.15.33:0/54688147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f0117fff700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01180c8e50 0x7f01181f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f011ca4b700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011815dd40 0x7f01181eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 -- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01180c8e50 msgr2=0x7f01181f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01180c8e50 0x7f01181f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 -- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011815dd40 msgr2=0x7f01181eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011815dd40 0x7f01181eef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.602+0000 7f01177fe700 1 -- 172.21.15.33:0/54688147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080094a0 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f01177fe700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0100000c00 tx=0x7f0100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100004480 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010001f070 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0100009550 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01000038c0 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.603+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01180cd420 con 0x7f0118154520 2024-05-08T23:32:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.604+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01181f3dd0 con 0x7f0118154520 2024-05-08T23:32:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.604+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f0100021030 con 0x7f0118154520 2024-05-08T23:32:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.605+0000 7f0114ff9700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 0x7f00f806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.605+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f010008a300 con 0x7f0118154520 2024-05-08T23:32:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.605+0000 7f0117fff700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 0x7f00f806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.605+0000 7f0117fff700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 0x7f00f806d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d70 tx=0x7f0108009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.610+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f01181f3dd0 con 0x7f0118154520 2024-05-08T23:32:06.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.877+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01181f3dd0 con 0x7f0118154520 2024-05-08T23:32:06.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.877+0000 7f0114ff9700 1 -- 172.21.15.33:0/54688147 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5033 (secure 0 0 0) 0x7f01181f3dd0 con 0x7f0118154520 2024-05-08T23:32:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:06.877 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 msgr2=0x7f00f806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 0x7f00f806d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d70 tx=0x7f0108009210 comp rx=0 tx=0).stop 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 msgr2=0x7f01181ee9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0100000c00 tx=0x7f0100004750 comp rx=0 tx=0).stop 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 shutdown_connections 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00f806ac00 0x7f00f806d0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f011815dd40 0x7f01181eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01180c8e50 0x7f01181f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 --2- 172.21.15.33:0/54688147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0118154520 0x7f01181ee9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:06.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.879+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 >> 172.21.15.33:0/54688147 conn(0x7f0118072230 msgr2=0x7f01180c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:06.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.880+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 shutdown_connections 2024-05-08T23:32:06.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:06.880+0000 7f011e4ae700 1 -- 172.21.15.33:0/54688147 wait complete. 2024-05-08T23:32:06.880 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-05-08T23:32:06.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.029+0000 7fa82fa36700 1 Processor -- start 2024-05-08T23:32:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 -- start start 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa82815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa82815e280 0x7fa828070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82806baa0 con 0x7fa828154520 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82815e730 con 0x7fa82815e280 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.030+0000 7fa82fa36700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82806bcf0 con 0x7fa8280c8e50 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa82815e280 0x7fa828070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82d7d2700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa828154520 0x7fa8280cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49428/0 (socket says 172.21.15.33:49428) 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82d7d2700 1 -- 172.21.15.33:0/4002678033 learned_addr learned my addr 172.21.15.33:0/4002678033 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82cfd1700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa82815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 -- 172.21.15.33:0/4002678033 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 msgr2=0x7fa82815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa82815dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 -- 172.21.15.33:0/4002678033 >> v1:172.21.15.33:6789/0 conn(0x7fa828154520 legacy=0x7fa8280cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 -- 172.21.15.33:0/4002678033 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82806bfd0 con 0x7fa82815e280 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa81ffff700 1 -- 172.21.15.33:0/4002678033 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150847565 0 0) 0x7fa82806baa0 con 0x7fa828154520 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa82dfd3700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa82815e280 0x7fa828070140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa820009d90 tx=0x7fa82000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daca66cad11f0898 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa81ffff700 1 -- 172.21.15.33:0/4002678033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000e070 con 0x7fa82815e280 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa81ffff700 1 -- 172.21.15.33:0/4002678033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82000bbd0 con 0x7fa82815e280 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.031+0000 7fa81ffff700 1 -- 172.21.15.33:0/4002678033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820012430 con 0x7fa82815e280 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 -- 172.21.15.33:0/4002678033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa82815e280 msgr2=0x7fa828070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa82815e280 0x7fa828070140 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa820009d90 tx=0x7fa82000b330 comp rx=0 tx=0).stop 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 -- 172.21.15.33:0/4002678033 shutdown_connections 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa82815e280 0x7fa828070140 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 --2- 172.21.15.33:0/4002678033 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa82815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 -- 172.21.15.33:0/4002678033 >> 172.21.15.33:0/4002678033 conn(0x7fa828072230 msgr2=0x7fa828072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 -- 172.21.15.33:0/4002678033 shutdown_connections 2024-05-08T23:32:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.032+0000 7fa82fa36700 1 -- 172.21.15.33:0/4002678033 wait complete. 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.034+0000 7fa82fa36700 1 Processor -- start 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.034+0000 7fa82fa36700 1 -- start start 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa828157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa82815e280 0x7fa8280cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8281f0380 con 0x7fa828154520 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82815d390 con 0x7fa82815e280 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82fa36700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8281ef720 con 0x7fa8280c8e50 2024-05-08T23:32:07.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48356/0 (socket says 172.21.15.33:48356) 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82d7d2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa828157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 -- 172.21.15.33:0/1552854589 learned_addr learned my addr 172.21.15.33:0/1552854589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82dfd3700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa82815e280 0x7fa8280cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 -- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 msgr2=0x7fa828157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa828157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 -- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa82815e280 msgr2=0x7fa8280cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa82815e280 0x7fa8280cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 -- 172.21.15.33:0/1552854589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820009a70 con 0x7fa828154520 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.035+0000 7fa82cfd1700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e7e0 tx=0x7fa81800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.036+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800c070 con 0x7fa828154520 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.036+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818013610 con 0x7fa828154520 2024-05-08T23:32:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.036+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8281ef9e0 con 0x7fa828154520 2024-05-08T23:32:07.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.036+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818013980 con 0x7fa828154520 2024-05-08T23:32:07.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.036+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8281effa0 con 0x7fa828154520 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.037+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8281f04c0 con 0x7fa828154520 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.037+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7fa818013ae0 con 0x7fa828154520 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.038+0000 7fa81dffb700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 0x7fa80c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.038+0000 7fa82d7d2700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 0x7fa80c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.038+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa818088b60 con 0x7fa828154520 2024-05-08T23:32:07.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.038+0000 7fa82d7d2700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 0x7fa80c06d0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa824007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.043+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa8281f04c0 con 0x7fa828154520 2024-05-08T23:32:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.276+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8281f04c0 con 0x7fa828154520 2024-05-08T23:32:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.277+0000 7fa81dffb700 1 -- 172.21.15.33:0/1552854589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12347 (secure 0 0 0) 0x7fa8281f04c0 con 0x7fa828154520 2024-05-08T23:32:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:07.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:06.707224+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 msgr2=0x7fa80c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 0x7fa80c06d0c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa824007de0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 msgr2=0x7fa8280cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa81800e7e0 tx=0x7fa81800eaf0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 shutdown_connections 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80c06ac00 0x7fa80c06d0c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa82815e280 0x7fa8280cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa8280c8e50 0x7fa828157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 --2- 172.21.15.33:0/1552854589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa828154520 0x7fa8280cd2b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 >> 172.21.15.33:0/1552854589 conn(0x7fa828072230 msgr2=0x7fa828153330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 shutdown_connections 2024-05-08T23:32:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.279+0000 7fa82fa36700 1 -- 172.21.15.33:0/1552854589 wait complete. 2024-05-08T23:32:07.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:07.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.438+0000 7f1972165700 1 Processor -- start 2024-05-08T23:32:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 -- start start 2024-05-08T23:32:07.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f196c15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f196c06f810 0x7f196c164bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c15d870 con 0x7f196c15ad80 2024-05-08T23:32:07.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c071150 con 0x7f196c06f810 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.439+0000 7f1972165700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c7e30 con 0x7f196c154460 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196affd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f196c15ad80 0x7f196c06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49432/0 (socket says 172.21.15.33:49432) 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196affd700 1 -- 172.21.15.33:0/3797669088 learned_addr learned my addr 172.21.15.33:0/3797669088 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196bfff700 1 --2- 172.21.15.33:0/3797669088 >> v2:172.21.15.42:3300/0 conn(0x7f196c06f810 0x7f196c164bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196b7fe700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f196c15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185928499 0 0) 0x7f196c15d870 con 0x7f196c15ad80 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19540035b0 con 0x7f196c15ad80 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272035174 0 0) 0x7f19540035b0 con 0x7f196c15ad80 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 msgr2=0x7f196c15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f196c15dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 >> v2:172.21.15.42:3300/0 conn(0x7f196c06f810 msgr2=0x7f196c164bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 --2- 172.21.15.33:0/3797669088 >> v2:172.21.15.42:3300/0 conn(0x7f196c06f810 0x7f196c164bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c15b220 con 0x7f196c15ad80 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f1960001670 con 0x7f196c15ad80 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196bfff700 1 --2- 172.21.15.33:0/3797669088 >> v2:172.21.15.42:3300/0 conn(0x7f196c06f810 0x7f196c164bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 >> v1:172.21.15.33:6789/0 conn(0x7f196c15ad80 legacy=0x7f196c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f1954003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1954003fd0 0x7f1954004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954004d90 0x7f1954005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c071150 con 0x7f1954003fd0 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c7e30 con 0x7f1954004d90 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.440+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195400ba10 con 0x7f196c154460 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1954003fd0 0x7f1954004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196bfff700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954004d90 0x7f1954005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196b7fe700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f1954003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 -- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 msgr2=0x7f1954003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c154460 0x7f1954003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 -- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954004d90 msgr2=0x7f1954005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954004d90 0x7f1954005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 -- 172.21.15.33:0/3797669088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540035b0 con 0x7f1954003fd0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196affd700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1954003fd0 0x7f1954004850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f196000c240 tx=0x7f1954005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393f2d21605bee34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000cd40 con 0x7f1954003fd0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960005960 con 0x7f1954003fd0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.441+0000 7f196a7fc700 1 -- 172.21.15.33:0/3797669088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000cea0 con 0x7f1954003fd0 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 -- 172.21.15.33:0/3797669088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1954003fd0 msgr2=0x7f1954004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 --2- 172.21.15.33:0/3797669088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1954003fd0 0x7f1954004850 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f196000c240 tx=0x7f1954005880 comp rx=0 tx=0).stop 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f196bfff700 1 -- 172.21.15.33:0/3797669088 reap_dead start 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 -- 172.21.15.33:0/3797669088 shutdown_connections 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 -- 172.21.15.33:0/3797669088 >> 172.21.15.33:0/3797669088 conn(0x7f196c06e990 msgr2=0x7f196c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 -- 172.21.15.33:0/3797669088 shutdown_connections 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.442+0000 7f1972165700 1 -- 172.21.15.33:0/3797669088 wait complete. 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.443+0000 7f1972165700 1 Processor -- start 2024-05-08T23:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.443+0000 7f1972165700 1 -- start start 2024-05-08T23:32:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954003fd0 0x7f196c1ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c151a60 0x7f196c0c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1f0db0 con 0x7f196c1e51d0 2024-05-08T23:32:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1e37b0 con 0x7f1954003fd0 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f1972165700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1e3a90 con 0x7f196c151a60 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f196affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f196affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48402/0 (socket says 172.21.15.33:48402) 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.444+0000 7f196affd700 1 -- 172.21.15.33:0/1586287855 learned_addr learned my addr 172.21.15.33:0/1586287855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196b7fe700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954003fd0 0x7f196c1ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196bfff700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c151a60 0x7f196c0c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 -- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c151a60 msgr2=0x7f196c0c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c151a60 0x7f196c0c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 -- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954003fd0 msgr2=0x7f196c1ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954003fd0 0x7f196c1ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 -- 172.21.15.33:0/1586287855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c1e3d70 con 0x7f196c1e51d0 2024-05-08T23:32:07.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f196affd700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1960015040 tx=0x7f1960000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960022070 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1960005d60 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c1e4050 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960004a70 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.445+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c15a460 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.446+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c075460 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.447+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f1960004bd0 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.447+0000 7f1953fff700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 0x7f195806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.447+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1960089d10 con 0x7f196c1e51d0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.447+0000 7f196b7fe700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 0x7f195806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.448+0000 7f196b7fe700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 0x7f195806d0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f195c009ac0 tx=0x7f195c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.452+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f19600553b0 con 0x7f196c1e51d0 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.674+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f196c0b2e10 con 0x7f196c1e51d0 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.674+0000 7f1953fff700 1 -- 172.21.15.33:0/1586287855 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19600589d0 con 0x7f196c1e51d0 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:32:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 msgr2=0x7f195806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 0x7f195806d0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f195c009ac0 tx=0x7f195c008040 comp rx=0 tx=0).stop 2024-05-08T23:32:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 msgr2=0x7f196c1e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1960015040 tx=0x7f1960000fe0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 shutdown_connections 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f195806ac00 0x7f195806d0c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f196c151a60 0x7f196c0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1954003fd0 0x7f196c1ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 --2- 172.21.15.33:0/1586287855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c1e51d0 0x7f196c1e3170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 >> 172.21.15.33:0/1586287855 conn(0x7f196c06e990 msgr2=0x7f196c06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 shutdown_connections 2024-05-08T23:32:07.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.677+0000 7f1972165700 1 -- 172.21.15.33:0/1586287855 wait complete. 2024-05-08T23:32:07.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:32:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.830+0000 7f02a98a8700 1 Processor -- start 2024-05-08T23:32:07.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.830+0000 7f02a98a8700 1 -- start start 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a98a8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f02a4154820 0x7f02a406c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a98a8700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a415a5f0 con 0x7f02a40ce410 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a98a8700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a415a8d0 con 0x7f02a40ce030 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a98a8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a415abb0 con 0x7f02a4154820 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a27fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f02a40ce410 0x7f02a4154110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49440/0 (socket says 172.21.15.33:49440) 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a27fc700 1 -- 172.21.15.33:0/839939157 learned_addr learned my addr 172.21.15.33:0/839939157 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a37fe700 1 --2- 172.21.15.33:0/839939157 >> v2:172.21.15.33:3300/0 conn(0x7f02a4154820 0x7f02a406c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225550819 0 0) 0x7f02a415a5f0 con 0x7f02a40ce410 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f028c0035b0 con 0x7f02a40ce410 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532136669 0 0) 0x7f02a415a8d0 con 0x7f02a40ce030 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a415a5f0 con 0x7f02a40ce030 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a37fe700 1 -- 172.21.15.33:0/839939157 >> v1:172.21.15.42:6789/0 conn(0x7f02a40ce030 legacy=0x7f02a4070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a37fe700 1 -- 172.21.15.33:0/839939157 >> v1:172.21.15.33:6789/0 conn(0x7f02a40ce410 legacy=0x7f02a4154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.831+0000 7f02a37fe700 1 -- 172.21.15.33:0/839939157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a415ae90 con 0x7f02a4154820 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.832+0000 7f02a37fe700 1 --2- 172.21.15.33:0/839939157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a406c4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab47842f1b76630 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.832+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e070 con 0x7f02a4154820 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.832+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00bae0 con 0x7f02a4154820 2024-05-08T23:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.832+0000 7f02a1ffb700 1 -- 172.21.15.33:0/839939157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00be50 con 0x7f02a4154820 2024-05-08T23:32:07.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 -- 172.21.15.33:0/839939157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 msgr2=0x7f02a406c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 --2- 172.21.15.33:0/839939157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a406c4d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f029c009d90 tx=0x7f029c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 -- 172.21.15.33:0/839939157 shutdown_connections 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 --2- 172.21.15.33:0/839939157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a406c4d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 -- 172.21.15.33:0/839939157 >> 172.21.15.33:0/839939157 conn(0x7f02a4072390 msgr2=0x7f02a4072770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 -- 172.21.15.33:0/839939157 shutdown_connections 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.833+0000 7f02a98a8700 1 -- 172.21.15.33:0/839939157 wait complete. 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.834+0000 7f02a98a8700 1 Processor -- start 2024-05-08T23:32:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 -- start start 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02a40ce030 0x7f02a4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f02a40ce410 0x7f02a40c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40cb280 con 0x7f02a4154820 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4154c30 con 0x7f02a40ce030 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a98a8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40cb540 con 0x7f02a40ce410 2024-05-08T23:32:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48422/0 (socket says 172.21.15.33:48422) 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a37fe700 1 -- 172.21.15.33:0/1655140999 learned_addr learned my addr 172.21.15.33:0/1655140999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.835+0000 7f02a2ffd700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02a40ce030 0x7f02a4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a27fc700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f02a40ce410 0x7f02a40c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 -- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f02a40ce410 msgr2=0x7f02a40c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f02a40ce410 0x7f02a40c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 -- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02a40ce030 msgr2=0x7f02a4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02a40ce030 0x7f02a4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 -- 172.21.15.33:0/1655140999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009a70 con 0x7f02a4154820 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a37fe700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f029c004000 tx=0x7f029c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0048f0 con 0x7f02a4154820 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c009410 con 0x7f02a4154820 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4154ef0 con 0x7f02a4154820 2024-05-08T23:32:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c025680 con 0x7f02a4154820 2024-05-08T23:32:07.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.836+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a4157350 con 0x7f02a4154820 2024-05-08T23:32:07.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.837+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a4075460 con 0x7f02a4154820 2024-05-08T23:32:07.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.838+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f029c012430 con 0x7f02a4154820 2024-05-08T23:32:07.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.838+0000 7f028bfff700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 0x7f029006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:07.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.839+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f029c0923b0 con 0x7f02a4154820 2024-05-08T23:32:07.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.839+0000 7f02a2ffd700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 0x7f029006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:07.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.839+0000 7f02a2ffd700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 0x7f029006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0294000f60 tx=0x7f0294009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:07.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:07.843+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f029c055680 con 0x7f02a4154820 2024-05-08T23:32:08.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.065+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02a406c6b0 con 0x7f02a4154820 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.066+0000 7f028bfff700 1 -- 172.21.15.33:0/1655140999 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f029c0243c0 con 0x7f02a4154820 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-05-08T23:32:08.065 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-05-08T23:32:08.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:08.066 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 msgr2=0x7f029006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 0x7f029006d0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0294000f60 tx=0x7f0294009040 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 msgr2=0x7f02a414d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f029c004000 tx=0x7f029c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 shutdown_connections 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f029006ac00 0x7f029006d0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02a4154820 0x7f02a414d010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02a40ce030 0x7f02a4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 --2- 172.21.15.33:0/1655140999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f02a40ce410 0x7f02a40c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 >> 172.21.15.33:0/1655140999 conn(0x7f02a4072390 msgr2=0x7f02a4151970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 shutdown_connections 2024-05-08T23:32:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.068+0000 7f02a98a8700 1 -- 172.21.15.33:0/1655140999 wait complete. 2024-05-08T23:32:08.081 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:32:08.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:32:08.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.223+0000 7f6483413700 1 Processor -- start 2024-05-08T23:32:08.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.223+0000 7f6483413700 1 -- start start 2024-05-08T23:32:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6483413700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6483413700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0c93b0 con 0x7f647c1548a0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6483413700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c15e920 con 0x7f647c1544c0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6483413700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0c7140 con 0x7f647c0c9ce0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64809ae700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f647c1548a0 0x7f647c0c95d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49452/0 (socket says 172.21.15.33:49452) 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64809ae700 1 -- 172.21.15.33:0/2353810007 learned_addr learned my addr 172.21.15.33:0/2353810007 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64819b0700 1 --2- 172.21.15.33:0/2353810007 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6473fff700 1 -- 172.21.15.33:0/2353810007 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408098029 0 0) 0x7f647c15e920 con 0x7f647c1544c0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f6473fff700 1 -- 172.21.15.33:0/2353810007 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64680035b0 con 0x7f647c1544c0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64819b0700 1 -- 172.21.15.33:0/2353810007 >> v1:172.21.15.42:6789/0 conn(0x7f647c1544c0 legacy=0x7f647c072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64819b0700 1 -- 172.21.15.33:0/2353810007 >> v1:172.21.15.33:6789/0 conn(0x7f647c1548a0 legacy=0x7f647c0c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.225+0000 7f64819b0700 1 -- 172.21.15.33:0/2353810007 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c15db00 con 0x7f647c0c9ce0 2024-05-08T23:32:08.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f64819b0700 1 --2- 172.21.15.33:0/2353810007 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c071160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6474009dc0 tx=0x7f647400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a4786e132b75c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f6473fff700 1 -- 172.21.15.33:0/2353810007 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647400e070 con 0x7f647c0c9ce0 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f6473fff700 1 -- 172.21.15.33:0/2353810007 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647400bb20 con 0x7f647c0c9ce0 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f6473fff700 1 -- 172.21.15.33:0/2353810007 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6474012430 con 0x7f647c0c9ce0 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f6483413700 1 -- 172.21.15.33:0/2353810007 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 msgr2=0x7f647c071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.226+0000 7f6483413700 1 --2- 172.21.15.33:0/2353810007 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c071160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6474009dc0 tx=0x7f647400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.227+0000 7f6483413700 1 -- 172.21.15.33:0/2353810007 shutdown_connections 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.227+0000 7f6483413700 1 --2- 172.21.15.33:0/2353810007 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c071160 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.227+0000 7f6483413700 1 -- 172.21.15.33:0/2353810007 >> 172.21.15.33:0/2353810007 conn(0x7f647c06eb40 msgr2=0x7f647c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.227+0000 7f6483413700 1 -- 172.21.15.33:0/2353810007 shutdown_connections 2024-05-08T23:32:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.227+0000 7f6483413700 1 -- 172.21.15.33:0/2353810007 wait complete. 2024-05-08T23:32:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.228+0000 7f6483413700 1 Processor -- start 2024-05-08T23:32:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.228+0000 7f6483413700 1 -- start start 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f647c1548a0 0x7f647c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0cc540 con 0x7f647c1544c0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c1f3a20 con 0x7f647c1548a0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.229+0000 7f6483413700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0cbf80 con 0x7f647c0c9ce0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48446/0 (socket says 172.21.15.33:48446) 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 -- 172.21.15.33:0/2787933922 learned_addr learned my addr 172.21.15.33:0/2787933922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64811af700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64819b0700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f647c1548a0 0x7f647c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 -- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 msgr2=0x7f647c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 -- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f647c1548a0 msgr2=0x7f647c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f647c1548a0 0x7f647c159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 -- 172.21.15.33:0/2787933922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009a70 con 0x7f647c1544c0 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64811af700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.230+0000 7f64809ae700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c4b0 tx=0x7f646c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:08.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.283+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c005d70 con 0x7f647c1544c0 2024-05-08T23:32:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.283+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c0cc260 con 0x7f647c1544c0 2024-05-08T23:32:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.283+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c00ce80 con 0x7f647c1544c0 2024-05-08T23:32:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.283+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c0072d0 con 0x7f647c1544c0 2024-05-08T23:32:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.283+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c1f36b0 con 0x7f647c1544c0 2024-05-08T23:32:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.284+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c1f3230 con 0x7f647c1544c0 2024-05-08T23:32:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.285+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f646c007430 con 0x7f647c1544c0 2024-05-08T23:32:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.285+0000 7f6471ffb700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 0x7f646006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:08.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.285+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f646c088080 con 0x7f647c1544c0 2024-05-08T23:32:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.285+0000 7f64811af700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 0x7f646006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:08.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.286+0000 7f64811af700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 0x7f646006d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64780011e0 tx=0x7f6478009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:08.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.290+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f647c1f3230 con 0x7f647c1544c0 2024-05-08T23:32:08.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.513+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f647c1f3870 con 0x7f647c1544c0 2024-05-08T23:32:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.866+0000 7f6471ffb700 1 -- 172.21.15.33:0/2787933922 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f646c053750 con 0x7f647c1544c0 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.871+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 msgr2=0x7f646006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.871+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 0x7f646006d0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64780011e0 tx=0x7f6478009040 comp rx=0 tx=0).stop 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.871+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 msgr2=0x7f647c1efcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.871+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c4b0 tx=0x7f646c00c7c0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 shutdown_connections 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f646006ac00 0x7f646006d0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f647c1548a0 0x7f647c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f647c0c9ce0 0x7f647c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 --2- 172.21.15.33:0/2787933922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f647c1544c0 0x7f647c1efcc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 >> 172.21.15.33:0/2787933922 conn(0x7f647c06eb40 msgr2=0x7f647c0c5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.872+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 shutdown_connections 2024-05-08T23:32:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:08.873+0000 7f6483413700 1 -- 172.21.15.33:0/2787933922 wait complete. 2024-05-08T23:32:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-05-08T23:32:08.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:32:09.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.036+0000 7f5c5687d700 1 Processor -- start 2024-05-08T23:32:09.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c5687d700 1 -- start start 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c5687d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5c5006d980 0x7f5c5015e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c5687d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c500c2860 con 0x7f5c50154540 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c5687d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c500c8620 con 0x7f5c500712f0 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c5687d700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500c3540 con 0x7f5c5006d980 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c4ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5c50154540 0x7f5c500c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49458/0 (socket says 172.21.15.33:49458) 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.037+0000 7f5c4ffff700 1 -- 172.21.15.33:0/1377633979 learned_addr learned my addr 172.21.15.33:0/1377633979 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334948337 0 0) 0x7f5c500c2860 con 0x7f5c50154540 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c54e1a700 1 --2- 172.21.15.33:0/1377633979 >> v2:172.21.15.42:3300/0 conn(0x7f5c5006d980 0x7f5c5015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c380035b0 con 0x7f5c50154540 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478017190 0 0) 0x7f5c380035b0 con 0x7f5c50154540 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 >> v2:172.21.15.42:3300/0 conn(0x7f5c5006d980 msgr2=0x7f5c5015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 --2- 172.21.15.33:0/1377633979 >> v2:172.21.15.42:3300/0 conn(0x7f5c5006d980 0x7f5c5015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 >> v1:172.21.15.42:6789/0 conn(0x7f5c500712f0 legacy=0x7f5c5015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500c2ae0 con 0x7f5c50154540 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f5c40001670 con 0x7f5c50154540 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 >> v1:172.21.15.33:6789/0 conn(0x7f5c50154540 legacy=0x7f5c500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c3540 con 0x7f5c38004850 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800c0b0 con 0x7f5c38003a00 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3800c270 con 0x7f5c38005210 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.038+0000 7f5c4ffff700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c54e1a700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 -- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c38005210 msgr2=0x7f5c3800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c38005210 0x7f5c3800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 -- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c38003a00 msgr2=0x7f5c38003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c38003a00 0x7f5c38003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 -- 172.21.15.33:0/1377633979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380035b0 con 0x7f5c38004850 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c54e1a700 1 -- 172.21.15.33:0/1377633979 reap_dead start 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c38004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c440017c0 tx=0x7f5c38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd25d9278b1b833 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440052d0 con 0x7f5c38004850 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44005430 con 0x7f5c38004850 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.039+0000 7f5c4effd700 1 -- 172.21.15.33:0/1377633979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440038c0 con 0x7f5c38004850 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 -- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 msgr2=0x7f5c38004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c38004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c440017c0 tx=0x7f5c38005880 comp rx=0 tx=0).stop 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 -- 172.21.15.33:0/1377633979 shutdown_connections 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 --2- 172.21.15.33:0/1377633979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c38004cd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 -- 172.21.15.33:0/1377633979 >> 172.21.15.33:0/1377633979 conn(0x7f5c5006ce60 msgr2=0x7f5c5006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 -- 172.21.15.33:0/1377633979 shutdown_connections 2024-05-08T23:32:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.040+0000 7f5c5687d700 1 -- 172.21.15.33:0/1377633979 wait complete. 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.041+0000 7f5c5687d700 1 Processor -- start 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.041+0000 7f5c5687d700 1 -- start start 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c5006bc50 0x7f5c501ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 0x7f5c50147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501f0300 con 0x7f5c38004850 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500ccbd0 con 0x7f5c5006bc50 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c5687d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500cc3d0 con 0x7f5c501efe80 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48470/0 (socket says 172.21.15.33:48470) 2024-05-08T23:32:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 -- 172.21.15.33:0/2223695076 learned_addr learned my addr 172.21.15.33:0/2223695076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c5006bc50 0x7f5c501ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c54e1a700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 0x7f5c50147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 -- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 msgr2=0x7f5c50147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 0x7f5c50147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 -- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c5006bc50 msgr2=0x7f5c501ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c5006bc50 0x7f5c501ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c4ffff700 1 -- 172.21.15.33:0/2223695076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5015d0c0 con 0x7f5c38004850 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.042+0000 7f5c54e1a700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 0x7f5c50147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c4ffff700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c400072c0 tx=0x7f5c400075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:09.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000f6c0 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4000f840 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5015d2f0 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000fbf0 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.043+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500cc8f0 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.044+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c50073770 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.044+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 89089+0+0 (secure 0 0 0) 0x7f5c4000fd50 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.045+0000 7f5c4cff9700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 0x7f5c3c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.045+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5c40088f90 con 0x7f5c38004850 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.045+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 0x7f5c3c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.045+0000 7f5c4f7fe700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 0x7f5c3c06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.050+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5c40054430 con 0x7f5c38004850 2024-05-08T23:32:09.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.269+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5c50075460 con 0x7f5c38004850 2024-05-08T23:32:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.729+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5c4005e0d0 con 0x7f5c38004850 2024-05-08T23:32:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.870+0000 7f5c4cff9700 1 -- 172.21.15.33:0/2223695076 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5c4006fed0 con 0x7f5c38004850 2024-05-08T23:32:09.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.874+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 msgr2=0x7f5c3c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.874+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 0x7f5c3c06d0c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5c44000c00 tx=0x7f5c44009030 comp rx=0 tx=0).stop 2024-05-08T23:32:09.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.874+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 msgr2=0x7f5c5006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.874+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c400072c0 tx=0x7f5c400075d0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 shutdown_connections 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5c3c06ac00 0x7f5c3c06d0c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5c501efe80 0x7f5c50147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c38004850 0x7f5c5006c670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 --2- 172.21.15.33:0/2223695076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c5006bc50 0x7f5c501ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.875+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 >> 172.21.15.33:0/2223695076 conn(0x7f5c5006ce60 msgr2=0x7f5c5014f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.876+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 shutdown_connections 2024-05-08T23:32:09.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:09.876+0000 7f5c5687d700 1 -- 172.21.15.33:0/2223695076 wait complete. 2024-05-08T23:32:09.878 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-05-08T23:32:09.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:32:10.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.044+0000 7f3eb3e35700 1 Processor -- start 2024-05-08T23:32:10.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 -- start start 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3eac15b360 0x7f3eac06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac0711c0 con 0x7f3eac0b1ad0 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac06e320 con 0x7f3eac15b360 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.045+0000 7f3eb3e35700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac071710 con 0x7f3eac0c9220 2024-05-08T23:32:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb13d0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb1bd1700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3eac15b360 0x7f3eac06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb13d0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:47954/0 (socket says 172.21.15.33:47954) 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb13d0700 1 -- 172.21.15.33:0/2098797321 learned_addr learned my addr 172.21.15.33:0/2098797321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb23d2700 1 --1- 172.21.15.33:0/2098797321 >> v1:172.21.15.42:6789/0 conn(0x7f3eac0b1ad0 0x7f3eac15dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:52536/0 (socket says 172.21.15.33:52536) 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb1bd1700 1 -- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 msgr2=0x7f3eac0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb1bd1700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb1bd1700 1 -- 172.21.15.33:0/2098797321 >> v1:172.21.15.42:6789/0 conn(0x7f3eac0b1ad0 legacy=0x7f3eac15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb1bd1700 1 -- 172.21.15.33:0/2098797321 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac071430 con 0x7f3eac15b360 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.046+0000 7f3eb13d0700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb1bd1700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac15b360 0x7f3eac06a990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009840 tx=0x7f3ea0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec34b5fb4080616 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb0bcf700 1 -- 172.21.15.33:0/2098797321 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011830 con 0x7f3eac15b360 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb0bcf700 1 -- 172.21.15.33:0/2098797321 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0011990 con 0x7f3eac15b360 2024-05-08T23:32:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb0bcf700 1 -- 172.21.15.33:0/2098797321 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0011d20 con 0x7f3eac15b360 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 -- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac15b360 msgr2=0x7f3eac06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac15b360 0x7f3eac06a990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009840 tx=0x7f3ea0009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 -- 172.21.15.33:0/2098797321 shutdown_connections 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac15b360 0x7f3eac06a990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/2098797321 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 -- 172.21.15.33:0/2098797321 >> 172.21.15.33:0/2098797321 conn(0x7f3eac06d2b0 msgr2=0x7f3eac06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 -- 172.21.15.33:0/2098797321 shutdown_connections 2024-05-08T23:32:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.047+0000 7f3eb3e35700 1 -- 172.21.15.33:0/2098797321 wait complete. 2024-05-08T23:32:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 Processor -- start 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 -- start start 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac0b1ad0 0x7f3eac152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.049+0000 7f3eb3e35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1efc30 con 0x7f3eac15b360 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb3e35700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac152730 con 0x7f3eac0b1ad0 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb3e35700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1efed0 con 0x7f3eac0c9220 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48492/0 (socket says 172.21.15.33:48492) 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 -- 172.21.15.33:0/3531792493 learned_addr learned my addr 172.21.15.33:0/3531792493 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb1bd1700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac0b1ad0 0x7f3eac152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb13d0700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 -- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 msgr2=0x7f3eac0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 -- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac0b1ad0 msgr2=0x7f3eac152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac0b1ad0 0x7f3eac152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 -- 172.21.15.33:0/3531792493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009550 con 0x7f3eac15b360 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb23d2700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cbc0 tx=0x7f3ea400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea400a9e0 con 0x7f3eac15b360 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea400ab40 con 0x7f3eac15b360 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00094a0 con 0x7f3eac15b360 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.050+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4007070 con 0x7f3eac15b360 2024-05-08T23:32:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.051+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac06b050 con 0x7f3eac15b360 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.052+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac075460 con 0x7f3eac15b360 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.052+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3ea40071d0 con 0x7f3eac15b360 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.052+0000 7f3e9e7fc700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 0x7f3e9006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.053+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3ea4088ef0 con 0x7f3eac15b360 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.053+0000 7f3eb1bd1700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 0x7f3e9006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.053+0000 7f3eb1bd1700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 0x7f3e9006d150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0000c00 tx=0x7f3ea001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.058+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ea4054150 con 0x7f3eac15b360 2024-05-08T23:32:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.329+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3eac068680 con 0x7f3eac15b360 2024-05-08T23:32:10.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.335+0000 7f3e9e7fc700 1 -- 172.21.15.33:0/3531792493 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3ea4017030 con 0x7f3eac15b360 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.341+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 msgr2=0x7f3e9006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.341+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 0x7f3e9006d150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0000c00 tx=0x7f3ea001c040 comp rx=0 tx=0).stop 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.341+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 msgr2=0x7f3eac0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.341+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cbc0 tx=0x7f3ea400ced0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 shutdown_connections 2024-05-08T23:32:10.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3e9006ac90 0x7f3e9006d150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac15b360 0x7f3eac0caa50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eac0b1ad0 0x7f3eac152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 --2- 172.21.15.33:0/3531792493 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3eac0c9220 0x7f3eac0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 >> 172.21.15.33:0/3531792493 conn(0x7f3eac06d2b0 msgr2=0x7f3eac06d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 shutdown_connections 2024-05-08T23:32:10.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.342+0000 7f3eb3e35700 1 -- 172.21.15.33:0/3531792493 wait complete. 2024-05-08T23:32:10.344 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:32:10.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.500+0000 7f2027e44700 1 Processor -- start 2024-05-08T23:32:10.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.500+0000 7f2027e44700 1 -- start start 2024-05-08T23:32:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2027e44700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 0x7f202006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2027e44700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f20200c87e0 0x7f2020161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2027e44700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20200c9590 con 0x7f20200ce380 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2027e44700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20200b3eb0 con 0x7f20200c87e0 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2027e44700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20200c9830 con 0x7f20200cdfa0 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f20253df700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f20200ce380 0x7f20200b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49470/0 (socket says 172.21.15.33:49470) 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f20253df700 1 -- 172.21.15.33:0/1747948124 learned_addr learned my addr 172.21.15.33:0/1747948124 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f20263e1700 1 --2- 172.21.15.33:0/1747948124 >> v2:172.21.15.42:3300/0 conn(0x7f20200c87e0 0x7f2020161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2025be0700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 0x7f202006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2024bde700 1 -- 172.21.15.33:0/1747948124 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448862988 0 0) 0x7f20200c9590 con 0x7f20200ce380 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.501+0000 7f2024bde700 1 -- 172.21.15.33:0/1747948124 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f200c0035b0 con 0x7f20200ce380 2024-05-08T23:32:10.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f20263e1700 1 -- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 msgr2=0x7f202006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f20263e1700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 0x7f202006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f20263e1700 1 -- 172.21.15.33:0/1747948124 >> v1:172.21.15.33:6789/0 conn(0x7f20200ce380 legacy=0x7f20200b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f20263e1700 1 -- 172.21.15.33:0/1747948124 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20200c9b10 con 0x7f20200c87e0 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f2025be0700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 0x7f202006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f20263e1700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200c87e0 0x7f2020161bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2018009d90 tx=0x7f201800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41864bdf29a65545 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f2024bde700 1 -- 172.21.15.33:0/1747948124 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800e070 con 0x7f20200c87e0 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f2024bde700 1 -- 172.21.15.33:0/1747948124 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201800bb20 con 0x7f20200c87e0 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.502+0000 7f2024bde700 1 -- 172.21.15.33:0/1747948124 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018012430 con 0x7f20200c87e0 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 -- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200c87e0 msgr2=0x7f2020161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200c87e0 0x7f2020161bd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2018009d90 tx=0x7f201800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 -- 172.21.15.33:0/1747948124 shutdown_connections 2024-05-08T23:32:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200c87e0 0x7f2020161bd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 --2- 172.21.15.33:0/1747948124 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200cdfa0 0x7f202006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 -- 172.21.15.33:0/1747948124 >> 172.21.15.33:0/1747948124 conn(0x7f2020072230 msgr2=0x7f2020072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 -- 172.21.15.33:0/1747948124 shutdown_connections 2024-05-08T23:32:10.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.503+0000 7f2027e44700 1 -- 172.21.15.33:0/1747948124 wait complete. 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.504+0000 7f2027e44700 1 Processor -- start 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.504+0000 7f2027e44700 1 -- start start 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200c87e0 0x7f2020158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200cdfa0 0x7f20201f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202015f430 con 0x7f20200ce380 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202015a790 con 0x7f20200cdfa0 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2027e44700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202015aa70 con 0x7f20200c87e0 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48494/0 (socket says 172.21.15.33:48494) 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 -- 172.21.15.33:0/318703597 learned_addr learned my addr 172.21.15.33:0/318703597 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f2025be0700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200c87e0 0x7f2020158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20253df700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200cdfa0 0x7f20201f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 -- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200c87e0 msgr2=0x7f2020158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200c87e0 0x7f2020158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 -- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200cdfa0 msgr2=0x7f20201f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200cdfa0 0x7f20201f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.505+0000 7f20263e1700 1 -- 172.21.15.33:0/318703597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0094a0 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f20263e1700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2018014040 tx=0x7f2018004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180129f0 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2018020070 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2018009a70 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018012dd0 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.506+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202015b030 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.507+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020075460 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.507+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2018004040 con 0x7f20200ce380 2024-05-08T23:32:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.508+0000 7f20167fc700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 0x7f200406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.508+0000 7f2025be0700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 0x7f200406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.508+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2018022070 con 0x7f20200ce380 2024-05-08T23:32:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.508+0000 7f2025be0700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 0x7f200406d150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f201c009d70 tx=0x7f201c009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.513+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f20180561a0 con 0x7f20200ce380 2024-05-08T23:32:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.728+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20201f37c0 con 0x7f20200ce380 2024-05-08T23:32:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.729+0000 7f20167fc700 1 -- 172.21.15.33:0/318703597 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7f201802c020 con 0x7f20200ce380 2024-05-08T23:32:10.729 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:10.729 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:10.330606+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 msgr2=0x7f200406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 0x7f200406d150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f201c009d70 tx=0x7f201c009210 comp rx=0 tx=0).stop 2024-05-08T23:32:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 msgr2=0x7f20201f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2018014040 tx=0x7f2018004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 shutdown_connections 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f200406ac90 0x7f200406d150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20200ce380 0x7f20201f4050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20200c87e0 0x7f2020158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 --2- 172.21.15.33:0/318703597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20200cdfa0 0x7f20201f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 >> 172.21.15.33:0/318703597 conn(0x7f2020072230 msgr2=0x7f2020072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 shutdown_connections 2024-05-08T23:32:10.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.732+0000 7f2027e44700 1 -- 172.21.15.33:0/318703597 wait complete. 2024-05-08T23:32:10.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.882+0000 7fbe0ed3f700 1 Processor -- start 2024-05-08T23:32:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.882+0000 7fbe0ed3f700 1 -- start start 2024-05-08T23:32:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0ed3f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe08151470 0x7fbe0806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0ed3f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fbe080c3880 0x7fbe0806c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0ed3f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe080cbc90 con 0x7fbe080c3330 2024-05-08T23:32:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0ed3f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe08151850 con 0x7fbe08151470 2024-05-08T23:32:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0ed3f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0815b510 con 0x7fbe080c3880 2024-05-08T23:32:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbe08151470 0x7fbe0806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe07fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe080c3330 0x7fbe0815a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49472/0 (socket says 172.21.15.33:49472) 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe07fff700 1 -- 172.21.15.33:0/1646140051 learned_addr learned my addr 172.21.15.33:0/1646140051 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0d2dc700 1 --2- 172.21.15.33:0/1646140051 >> v2:172.21.15.42:3300/0 conn(0x7fbe080c3880 0x7fbe0806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 -- 172.21.15.33:0/1646140051 >> v2:172.21.15.42:3300/0 conn(0x7fbe080c3880 msgr2=0x7fbe0806c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/1646140051 >> v2:172.21.15.42:3300/0 conn(0x7fbe080c3880 0x7fbe0806c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 -- 172.21.15.33:0/1646140051 >> v1:172.21.15.33:6789/0 conn(0x7fbe080c3330 legacy=0x7fbe0815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 -- 172.21.15.33:0/1646140051 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe080cb190 con 0x7fbe08151470 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.883+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/1646140051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08151470 0x7fbe0806bce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe08070e00 tx=0x7fbdfc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac77ed99396c426 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe077fe700 1 -- 172.21.15.33:0/1646140051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc011830 con 0x7fbe08151470 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe077fe700 1 -- 172.21.15.33:0/1646140051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc011990 con 0x7fbe08151470 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe077fe700 1 -- 172.21.15.33:0/1646140051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc011d20 con 0x7fbe08151470 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/1646140051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08151470 msgr2=0x7fbe0806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/1646140051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08151470 0x7fbe0806bce0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe08070e00 tx=0x7fbdfc009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/1646140051 shutdown_connections 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/1646140051 >> v2:172.21.15.42:3300/0 conn(0x7fbe080c3880 0x7fbe0806c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/1646140051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08151470 0x7fbe0806bce0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/1646140051 >> 172.21.15.33:0/1646140051 conn(0x7fbe080c9480 msgr2=0x7fbe080c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/1646140051 shutdown_connections 2024-05-08T23:32:10.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.884+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/1646140051 wait complete. 2024-05-08T23:32:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.886+0000 7fbe0ed3f700 1 Processor -- start 2024-05-08T23:32:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.886+0000 7fbe0ed3f700 1 -- start start 2024-05-08T23:32:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 0x7fbe081eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe08151470 0x7fbe081f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0814e510 con 0x7fbe080c3880 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081ee860 con 0x7fbe08151470 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe0ed3f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081f25c0 con 0x7fbe080c3330 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48508/0 (socket says 172.21.15.33:48508) 2024-05-08T23:32:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.887+0000 7fbe07fff700 1 -- 172.21.15.33:0/2844913704 learned_addr learned my addr 172.21.15.33:0/2844913704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe0d2dc700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe08151470 0x7fbe081f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 0x7fbe081eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 -- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 msgr2=0x7fbe081eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 0x7fbe081eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 -- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe08151470 msgr2=0x7fbe081f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe08151470 0x7fbe081f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 -- 172.21.15.33:0/2844913704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc0094a0 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 0x7fbe081eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe07fff700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c4b0 tx=0x7fbdf400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4005d70 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf40070d0 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe081e2f90 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4007440 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.888+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe081e29d0 con 0x7fbe080c3880 2024-05-08T23:32:10.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.889+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe081eea10 con 0x7fbe080c3880 2024-05-08T23:32:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.890+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbdf40075a0 con 0x7fbe080c3880 2024-05-08T23:32:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.890+0000 7fbe057fa700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 0x7fbdec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.890+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdf4088670 con 0x7fbe080c3880 2024-05-08T23:32:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.890+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 0x7fbdec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.891+0000 7fbe0cadb700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 0x7fbdec06d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0090b0 tx=0x7fbdfc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:10.895+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbdf4053900 con 0x7fbe080c3880 2024-05-08T23:32:11.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.116+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe08150c70 con 0x7fbe080c3880 2024-05-08T23:32:11.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.117+0000 7fbe057fa700 1 -- 172.21.15.33:0/2844913704 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7fbdf4056f20 con 0x7fbe080c3880 2024-05-08T23:32:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:11.117 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:10.330606+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 msgr2=0x7fbdec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 0x7fbdec06d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0090b0 tx=0x7fbdfc008030 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 msgr2=0x7fbe0814dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c4b0 tx=0x7fbdf400c870 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 shutdown_connections 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbdec06ac90 0x7fbdec06d150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe08151470 0x7fbe081f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe080c3330 0x7fbe081eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 --2- 172.21.15.33:0/2844913704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080c3880 0x7fbe0814dfd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 >> 172.21.15.33:0/2844913704 conn(0x7fbe080c9480 msgr2=0x7fbe08072a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 shutdown_connections 2024-05-08T23:32:11.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.120+0000 7fbe0ed3f700 1 -- 172.21.15.33:0/2844913704 wait complete. 2024-05-08T23:32:11.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:32:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.276+0000 7f06f5985700 1 Processor -- start 2024-05-08T23:32:11.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.276+0000 7f06f5985700 1 -- start start 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06f5985700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06f00c2180 0x7f06f00c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06f5985700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f0070010 con 0x7f06f006b660 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06f5985700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f0070870 con 0x7f06f00c8e50 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06f5985700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f00c2aa0 con 0x7f06f00c2180 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06eeffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06f00c2180 0x7f06f00c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06ef7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f06f006b660 0x7f06f0071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49476/0 (socket says 172.21.15.33:49476) 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06ef7fe700 1 -- 172.21.15.33:0/708730476 learned_addr learned my addr 172.21.15.33:0/708730476 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06eeffd700 1 -- 172.21.15.33:0/708730476 >> v1:172.21.15.42:6789/0 conn(0x7f06f00c8e50 legacy=0x7f06f0070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06eeffd700 1 -- 172.21.15.33:0/708730476 >> v1:172.21.15.33:6789/0 conn(0x7f06f006b660 legacy=0x7f06f0071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06edffb700 1 -- 172.21.15.33:0/708730476 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560874424 0 0) 0x7f06f0070010 con 0x7f06f006b660 2024-05-08T23:32:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06eeffd700 1 -- 172.21.15.33:0/708730476 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f015a820 con 0x7f06f00c2180 2024-05-08T23:32:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06eeffd700 1 --2- 172.21.15.33:0/708730476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f00c2560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009840 tx=0x7f06e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d994519bcaddc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06edffb700 1 -- 172.21.15.33:0/708730476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0011830 con 0x7f06f00c2180 2024-05-08T23:32:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.277+0000 7f06edffb700 1 -- 172.21.15.33:0/708730476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e0011990 con 0x7f06f00c2180 2024-05-08T23:32:11.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06edffb700 1 -- 172.21.15.33:0/708730476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0011d20 con 0x7f06f00c2180 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 -- 172.21.15.33:0/708730476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 msgr2=0x7f06f00c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 --2- 172.21.15.33:0/708730476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f00c2560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009840 tx=0x7f06e0009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 -- 172.21.15.33:0/708730476 shutdown_connections 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 --2- 172.21.15.33:0/708730476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f00c2560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 -- 172.21.15.33:0/708730476 >> 172.21.15.33:0/708730476 conn(0x7f06f00c8360 msgr2=0x7f06f00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 -- 172.21.15.33:0/708730476 shutdown_connections 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.278+0000 7f06f5985700 1 -- 172.21.15.33:0/708730476 wait complete. 2024-05-08T23:32:11.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.279+0000 7f06f5985700 1 Processor -- start 2024-05-08T23:32:11.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 -- start start 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 0x7f06f00c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06f00c8e50 0x7f06f0140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f015a540 con 0x7f06f00c2180 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f0159580 con 0x7f06f006b660 2024-05-08T23:32:11.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.280+0000 7f06f5985700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f0158e60 con 0x7f06f00c8e50 2024-05-08T23:32:11.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48530/0 (socket says 172.21.15.33:48530) 2024-05-08T23:32:11.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 -- 172.21.15.33:0/2482939113 learned_addr learned my addr 172.21.15.33:0/2482939113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:11.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06eeffd700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 0x7f06f00c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ef7fe700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06f00c8e50 0x7f06f0140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 -- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06f00c8e50 msgr2=0x7f06f0140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06f00c8e50 0x7f06f0140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 -- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 msgr2=0x7f06f00c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 0x7f06f00c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 -- 172.21.15.33:0/2482939113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e00094a0 con 0x7f06f00c2180 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06eeffd700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 0x7f06f00c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06ee7fc700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c4e0 tx=0x7f06e400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40052d0 con 0x7f06f00c2180 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e4005430 con 0x7f06f00c2180 2024-05-08T23:32:11.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f01597d0 con 0x7f06f00c2180 2024-05-08T23:32:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e40038c0 con 0x7f06f00c2180 2024-05-08T23:32:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.281+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f0158810 con 0x7f06f00c2180 2024-05-08T23:32:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.283+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f0075460 con 0x7f06f00c2180 2024-05-08T23:32:11.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.283+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f06e4003a20 con 0x7f06f00c2180 2024-05-08T23:32:11.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.283+0000 7f06d7fff700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 0x7f06dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.284+0000 7f06eeffd700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 0x7f06dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.284+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06e40889d0 con 0x7f06f00c2180 2024-05-08T23:32:11.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.284+0000 7f06eeffd700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 0x7f06dc06d180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009790 tx=0x7f06e001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.288+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f06e4053a00 con 0x7f06f00c2180 2024-05-08T23:32:11.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:11.558+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f06f0158fa0 con 0x7f06f00c2180 2024-05-08T23:32:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.341+0000 7f06d7fff700 1 -- 172.21.15.33:0/2482939113 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f06e4057020 con 0x7f06f00c2180 2024-05-08T23:32:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.344+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 msgr2=0x7f06dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.344+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 0x7f06dc06d180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009790 tx=0x7f06e001c040 comp rx=0 tx=0).stop 2024-05-08T23:32:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.344+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 msgr2=0x7f06f01f7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.344+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c4e0 tx=0x7f06e400c8a0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 shutdown_connections 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f06dc06acc0 0x7f06dc06d180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06f00c8e50 0x7f06f0140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06f006b660 0x7f06f00c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 --2- 172.21.15.33:0/2482939113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06f00c2180 0x7f06f01f7d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 >> 172.21.15.33:0/2482939113 conn(0x7f06f00c8360 msgr2=0x7f06f0071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 shutdown_connections 2024-05-08T23:32:12.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.345+0000 7f06f5985700 1 -- 172.21.15.33:0/2482939113 wait complete. 2024-05-08T23:32:12.359 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-05-08T23:32:12.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T23:32:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.504+0000 7f50406b9700 1 Processor -- start 2024-05-08T23:32:12.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.504+0000 7f50406b9700 1 -- start start 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f50406b9700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5038154520 0x7f50380cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f50406b9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f50406b9700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503806baa0 con 0x7f50380c8e50 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f50406b9700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503806bcf0 con 0x7f5038154520 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f50406b9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503806bfd0 con 0x7f503815e450 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503ec56700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503ec56700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:48012/0 (socket says 172.21.15.33:48012) 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503ec56700 1 -- 172.21.15.33:0/3503080574 learned_addr learned my addr 172.21.15.33:0/3503080574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503e455700 1 --2- 172.21.15.33:0/3503080574 >> v2:172.21.15.42:3300/0 conn(0x7f5038154520 0x7f50380cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503ec56700 1 -- 172.21.15.33:0/3503080574 >> v2:172.21.15.42:3300/0 conn(0x7f5038154520 msgr2=0x7f50380cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.505+0000 7f503ec56700 1 --2- 172.21.15.33:0/3503080574 >> v2:172.21.15.42:3300/0 conn(0x7f5038154520 0x7f50380cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503ec56700 1 -- 172.21.15.33:0/3503080574 >> v1:172.21.15.42:6789/0 conn(0x7f50380c8e50 legacy=0x7f503815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503ec56700 1 -- 172.21.15.33:0/3503080574 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503806c2b0 con 0x7f503815e450 2024-05-08T23:32:12.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503ec56700 1 --2- 172.21.15.33:0/3503080574 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5030009e50 tx=0x7f503000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee0926c77d992d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503d453700 1 -- 172.21.15.33:0/3503080574 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000e070 con 0x7f503815e450 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503d453700 1 -- 172.21.15.33:0/3503080574 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503000b910 con 0x7f503815e450 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f503d453700 1 -- 172.21.15.33:0/3503080574 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503000bcc0 con 0x7f503815e450 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f50406b9700 1 -- 172.21.15.33:0/3503080574 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 msgr2=0x7f50380ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.506+0000 7f50406b9700 1 --2- 172.21.15.33:0/3503080574 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5030009e50 tx=0x7f503000b330 comp rx=0 tx=0).stop 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 -- 172.21.15.33:0/3503080574 shutdown_connections 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 --2- 172.21.15.33:0/3503080574 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f50380ce270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 --2- 172.21.15.33:0/3503080574 >> v2:172.21.15.42:3300/0 conn(0x7f5038154520 0x7f50380cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 -- 172.21.15.33:0/3503080574 >> 172.21.15.33:0/3503080574 conn(0x7f5038072230 msgr2=0x7f5038072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 -- 172.21.15.33:0/3503080574 shutdown_connections 2024-05-08T23:32:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.507+0000 7f50406b9700 1 -- 172.21.15.33:0/3503080574 wait complete. 2024-05-08T23:32:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.508+0000 7f50406b9700 1 Processor -- start 2024-05-08T23:32:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.508+0000 7f50406b9700 1 -- start start 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5038154520 0x7f503814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f503814acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50381599d0 con 0x7f50380c8e50 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503815a8e0 con 0x7f5038154520 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f50406b9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5038158630 con 0x7f503815e450 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48542/0 (socket says 172.21.15.33:48542) 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 -- 172.21.15.33:0/3801068935 learned_addr learned my addr 172.21.15.33:0/3801068935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503ec56700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f503814acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503dc54700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5038154520 0x7f503814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 -- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 msgr2=0x7f503814acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f503814acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 -- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5038154520 msgr2=0x7f503814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5038154520 0x7f503814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 -- 172.21.15.33:0/3801068935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030009b30 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f503e455700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f503400e9b0 tx=0x7f503400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.509+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034009710 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.510+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400ee50 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.510+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503815a040 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.510+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400ca50 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.510+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503815a600 con 0x7f50380c8e50 2024-05-08T23:32:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.510+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50381f3030 con 0x7f50380c8e50 2024-05-08T23:32:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.511+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f503400cbb0 con 0x7f50380c8e50 2024-05-08T23:32:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.512+0000 7f502effd700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 0x7f501c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.512+0000 7f503dc54700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 0x7f501c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.512+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50340892a0 con 0x7f50380c8e50 2024-05-08T23:32:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.512+0000 7f503dc54700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 0x7f501c06d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50381f4060 tx=0x7f5028009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.517+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5034054480 con 0x7f50380c8e50 2024-05-08T23:32:12.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:12.784+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5038152350 con 0x7f50380c8e50 2024-05-08T23:32:13.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.346+0000 7f502effd700 1 -- 172.21.15.33:0/3801068935 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f503400ce60 con 0x7f50380c8e50 2024-05-08T23:32:13.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 msgr2=0x7f501c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 0x7f501c06d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f50381f4060 tx=0x7f5028009040 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 msgr2=0x7f5038152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f503400e9b0 tx=0x7f503400ecc0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 shutdown_connections 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f501c06ac90 0x7f501c06d150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f503815e450 0x7f503814acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50380c8e50 0x7f5038152780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 --2- 172.21.15.33:0/3801068935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5038154520 0x7f503814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 >> 172.21.15.33:0/3801068935 conn(0x7f5038072230 msgr2=0x7f50380c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 shutdown_connections 2024-05-08T23:32:13.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.349+0000 7f50406b9700 1 -- 172.21.15.33:0/3801068935 wait complete. 2024-05-08T23:32:13.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T23:32:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.508+0000 7f1952afe700 1 Processor -- start 2024-05-08T23:32:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.508+0000 7f1952afe700 1 -- start start 2024-05-08T23:32:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f1952afe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f194c0c2e90 0x7f194c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f1952afe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f1952afe700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c06e3c0 con 0x7f194c1522f0 2024-05-08T23:32:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f1952afe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0c9cf0 con 0x7f194c0c2e90 2024-05-08T23:32:13.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f1952afe700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c06f2b0 con 0x7f194c0c92d0 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f194bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f194c0c2e90 0x7f194c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f194bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f194c0c2e90 0x7f194c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48552/0 (socket says 172.21.15.33:48552) 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.509+0000 7f194bfff700 1 -- 172.21.15.33:0/2886767002 learned_addr learned my addr 172.21.15.33:0/2886767002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f195109b700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194bfff700 1 -- 172.21.15.33:0/2886767002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 msgr2=0x7f194c0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194bfff700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194bfff700 1 -- 172.21.15.33:0/2886767002 >> v1:172.21.15.42:6789/0 conn(0x7f194c1522f0 legacy=0x7f194c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194bfff700 1 -- 172.21.15.33:0/2886767002 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c06ecf0 con 0x7f194c0c2e90 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194bfff700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c0c2e90 0x7f194c0ce3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1938009920 tx=0x7f1938009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d756f7123f955039 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194b7fe700 1 -- 172.21.15.33:0/2886767002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800ba20 con 0x7f194c0c2e90 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194b7fe700 1 -- 172.21.15.33:0/2886767002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800bb80 con 0x7f194c0c2e90 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.510+0000 7f194b7fe700 1 -- 172.21.15.33:0/2886767002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938009e70 con 0x7f194c0c2e90 2024-05-08T23:32:13.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 -- 172.21.15.33:0/2886767002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c0c2e90 msgr2=0x7f194c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c0c2e90 0x7f194c0ce3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1938009920 tx=0x7f1938009c30 comp rx=0 tx=0).stop 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 -- 172.21.15.33:0/2886767002 shutdown_connections 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 --2- 172.21.15.33:0/2886767002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c0c2e90 0x7f194c0ce3d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 -- 172.21.15.33:0/2886767002 >> 172.21.15.33:0/2886767002 conn(0x7f194c066ae0 msgr2=0x7f194c066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 -- 172.21.15.33:0/2886767002 shutdown_connections 2024-05-08T23:32:13.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.511+0000 7f1952afe700 1 -- 172.21.15.33:0/2886767002 wait complete. 2024-05-08T23:32:13.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.512+0000 7f1952afe700 1 Processor -- start 2024-05-08T23:32:13.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.512+0000 7f1952afe700 1 -- start start 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f194c0c2e90 0x7f194c147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c06c8c0 con 0x7f194c1522f0 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c1536a0 con 0x7f194c0c2e90 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f1952afe700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b3360 con 0x7f194c0c92d0 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48556/0 (socket says 172.21.15.33:48556) 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 -- 172.21.15.33:0/819016252 learned_addr learned my addr 172.21.15.33:0/819016252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f194bfff700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195089a700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f194c0c2e90 0x7f194c147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 -- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 msgr2=0x7f194c153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 -- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f194c0c2e90 msgr2=0x7f194c147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f194c0c2e90 0x7f194c147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.513+0000 7f195109b700 1 -- 172.21.15.33:0/819016252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1938009550 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f195109b700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1944007e40 tx=0x7f194400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400ecd0 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400ee50 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944014660 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f1952afe700 1 -- 172.21.15.33:0/819016252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c06cce0 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.514+0000 7f1952afe700 1 -- 172.21.15.33:0/819016252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c06bf20 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.515+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c075460 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.515+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f19440147c0 con 0x7f194c1522f0 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.516+0000 7f19497fa700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.516+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1944089940 con 0x7f194c1522f0 2024-05-08T23:32:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.516+0000 7f195089a700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.516+0000 7f195089a700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 0x7f193006d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1940007fd0 tx=0x7f1940009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.520+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1944054bd0 con 0x7f194c1522f0 2024-05-08T23:32:13.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:13.792+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f194c155aa0 con 0x7f194c1522f0 2024-05-08T23:32:14.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.347+0000 7f19497fa700 1 -- 172.21.15.33:0/819016252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v7) v1 ==== 115+0+0 (secure 0 0 0) 0x7f19440581f0 con 0x7f194c1522f0 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.351+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 msgr2=0x7f193006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.351+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 0x7f193006d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1940007fd0 tx=0x7f1940009040 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.351+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 msgr2=0x7f194c0c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.351+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1944007e40 tx=0x7f194400e3c0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 shutdown_connections 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f193006ac90 0x7f193006d150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f194c1522f0 0x7f194c0c5480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f194c0c2e90 0x7f194c147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 --2- 172.21.15.33:0/819016252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f194c0c92d0 0x7f194c153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 >> 172.21.15.33:0/819016252 conn(0x7f194c066ae0 msgr2=0x7f194c14f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 shutdown_connections 2024-05-08T23:32:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.352+0000 7f1936ffd700 1 -- 172.21.15.33:0/819016252 wait complete. 2024-05-08T23:32:14.368 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T23:32:14.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.511+0000 7f27957cb700 1 Processor -- start 2024-05-08T23:32:14.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.511+0000 7f27957cb700 1 -- start start 2024-05-08T23:32:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f27957cb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27900c9ad0 0x7f2790070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f27957cb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279015b140 con 0x7f27901542f0 2024-05-08T23:32:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f27957cb700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279015a5c0 con 0x7f27901546d0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f27957cb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279015a8a0 con 0x7f27900c9ad0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f278f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27900c9ad0 0x7f2790070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.512+0000 7f278effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f27901542f0 0x7f27900cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49486/0 (socket says 172.21.15.33:49486) 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278effd700 1 -- 172.21.15.33:0/2215914119 learned_addr learned my addr 172.21.15.33:0/2215914119 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278f7fe700 1 -- 172.21.15.33:0/2215914119 >> v1:172.21.15.42:6789/0 conn(0x7f27901546d0 legacy=0x7f27900c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278f7fe700 1 -- 172.21.15.33:0/2215914119 >> v1:172.21.15.33:6789/0 conn(0x7f27901542f0 legacy=0x7f27900cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278f7fe700 1 -- 172.21.15.33:0/2215914119 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279015ab80 con 0x7f27900c9ad0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278dffb700 1 -- 172.21.15.33:0/2215914119 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817551878 0 0) 0x7f279015b140 con 0x7f27901542f0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278f7fe700 1 --2- 172.21.15.33:0/2215914119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790070180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2788009d90 tx=0x7f278800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c644de628db80bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278dffb700 1 -- 172.21.15.33:0/2215914119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278800e070 con 0x7f27900c9ad0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278dffb700 1 -- 172.21.15.33:0/2215914119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278800bbd0 con 0x7f27900c9ad0 2024-05-08T23:32:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.513+0000 7f278dffb700 1 -- 172.21.15.33:0/2215914119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788012430 con 0x7f27900c9ad0 2024-05-08T23:32:14.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 -- 172.21.15.33:0/2215914119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 msgr2=0x7f2790070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 --2- 172.21.15.33:0/2215914119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790070180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2788009d90 tx=0x7f278800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 -- 172.21.15.33:0/2215914119 shutdown_connections 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 --2- 172.21.15.33:0/2215914119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790070180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 -- 172.21.15.33:0/2215914119 >> 172.21.15.33:0/2215914119 conn(0x7f2790072230 msgr2=0x7f2790072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 -- 172.21.15.33:0/2215914119 shutdown_connections 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.514+0000 7f27957cb700 1 -- 172.21.15.33:0/2215914119 wait complete. 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.515+0000 7f27957cb700 1 Processor -- start 2024-05-08T23:32:14.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.515+0000 7f27957cb700 1 -- start start 2024-05-08T23:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 0x7f27900c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27901546d0 0x7f27900c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27901f3760 con 0x7f27900c9ad0 2024-05-08T23:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2790154ad0 con 0x7f27901542f0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f27957cb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27900c3a20 con 0x7f27901546d0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f278effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f278effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48570/0 (socket says 172.21.15.33:48570) 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f278effd700 1 -- 172.21.15.33:0/1238458628 learned_addr learned my addr 172.21.15.33:0/1238458628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f278e7fc700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 0x7f27900c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.516+0000 7f278f7fe700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27901546d0 0x7f27900c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 -- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27901546d0 msgr2=0x7f27900c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27901546d0 0x7f27900c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 -- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 msgr2=0x7f27900c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 0x7f27900c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 -- 172.21.15.33:0/1238458628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788009a70 con 0x7f27900c9ad0 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278e7fc700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 0x7f27900c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f278effd700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f278000c550 tx=0x7f278000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780004c80 con 0x7f27900c9ad0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780004e00 con 0x7f27900c9ad0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780007310 con 0x7f27900c9ad0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27900c3030 con 0x7f27900c9ad0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.517+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2790154d20 con 0x7f27900c9ad0 2024-05-08T23:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.518+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27900c3550 con 0x7f27900c9ad0 2024-05-08T23:32:14.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.519+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2780007470 con 0x7f27900c9ad0 2024-05-08T23:32:14.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.519+0000 7f27777fe700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 0x7f277c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.519+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2780088fc0 con 0x7f27900c9ad0 2024-05-08T23:32:14.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.519+0000 7f278e7fc700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 0x7f277c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.519+0000 7f278e7fc700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 0x7f277c06d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27901ece50 tx=0x7f2784009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.524+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f27800bc900 con 0x7f27900c9ad0 2024-05-08T23:32:14.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.790+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2790075460 con 0x7f27900c9ad0 2024-05-08T23:32:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.791+0000 7f27777fe700 1 -- 172.21.15.33:0/1238458628 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6240 (secure 0 0 0) 0x7f27800541a0 con 0x7f27900c9ad0 2024-05-08T23:32:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:14.790 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:14.344504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:14.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 msgr2=0x7f277c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 0x7f277c06d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27901ece50 tx=0x7f2784009040 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 msgr2=0x7f2790152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f278000c550 tx=0x7f278000c860 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 shutdown_connections 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f277c06ac90 0x7f277c06d150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27901546d0 0x7f27900c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27900c9ad0 0x7f2790152d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 --2- 172.21.15.33:0/1238458628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27901542f0 0x7f27900c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 >> 172.21.15.33:0/1238458628 conn(0x7f2790072230 msgr2=0x7f279006fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 shutdown_connections 2024-05-08T23:32:14.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.793+0000 7f27957cb700 1 -- 172.21.15.33:0/1238458628 wait complete. 2024-05-08T23:32:14.794 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-05-08T23:32:14.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.944+0000 7fd11b297700 1 Processor -- start 2024-05-08T23:32:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.944+0000 7fd11b297700 1 -- start start 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd11b297700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1140ce380 0x7fd114153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd11b297700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd1140c9ad0 0x7fd114154430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd11b297700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11415b140 con 0x7fd1140cdfa0 2024-05-08T23:32:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd11b297700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd114154970 con 0x7fd1140ce380 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd11b297700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1140c8f30 con 0x7fd1140c9ad0 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd118832700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd1140ce380 0x7fd114153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd119033700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd1140cdfa0 0x7fd1140c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49496/0 (socket says 172.21.15.33:49496) 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd119033700 1 -- 172.21.15.33:0/4139268100 learned_addr learned my addr 172.21.15.33:0/4139268100 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd118832700 1 --2- 172.21.15.33:0/4139268100 >> v2:172.21.15.33:3300/0 conn(0x7fd1140ce380 0x7fd114153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48582/0 (socket says 172.21.15.33:48582) 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.945+0000 7fd119834700 1 --2- 172.21.15.33:0/4139268100 >> v2:172.21.15.42:3300/0 conn(0x7fd1140c9ad0 0x7fd114154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd118832700 1 -- 172.21.15.33:0/4139268100 >> v2:172.21.15.42:3300/0 conn(0x7fd1140c9ad0 msgr2=0x7fd114154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd118832700 1 --2- 172.21.15.33:0/4139268100 >> v2:172.21.15.42:3300/0 conn(0x7fd1140c9ad0 0x7fd114154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd118832700 1 -- 172.21.15.33:0/4139268100 >> v1:172.21.15.33:6789/0 conn(0x7fd1140cdfa0 legacy=0x7fd1140c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd118832700 1 -- 172.21.15.33:0/4139268100 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11415a5c0 con 0x7fd1140ce380 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd10bfff700 1 -- 172.21.15.33:0/4139268100 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298738606 0 0) 0x7fd11415b140 con 0x7fd1140cdfa0 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd118832700 1 --2- 172.21.15.33:0/4139268100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140ce380 0x7fd114153ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd11406db50 tx=0x7fd104009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c224829acd6627ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd10bfff700 1 -- 172.21.15.33:0/4139268100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104011830 con 0x7fd1140ce380 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd10bfff700 1 -- 172.21.15.33:0/4139268100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104011990 con 0x7fd1140ce380 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.946+0000 7fd10bfff700 1 -- 172.21.15.33:0/4139268100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104011d20 con 0x7fd1140ce380 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 -- 172.21.15.33:0/4139268100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140ce380 msgr2=0x7fd114153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 --2- 172.21.15.33:0/4139268100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140ce380 0x7fd114153ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd11406db50 tx=0x7fd104009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 -- 172.21.15.33:0/4139268100 shutdown_connections 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 --2- 172.21.15.33:0/4139268100 >> v2:172.21.15.42:3300/0 conn(0x7fd1140c9ad0 0x7fd114154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 --2- 172.21.15.33:0/4139268100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140ce380 0x7fd114153ef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 -- 172.21.15.33:0/4139268100 >> 172.21.15.33:0/4139268100 conn(0x7fd114072230 msgr2=0x7fd114072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 -- 172.21.15.33:0/4139268100 shutdown_connections 2024-05-08T23:32:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.947+0000 7fd11b297700 1 -- 172.21.15.33:0/4139268100 wait complete. 2024-05-08T23:32:14.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.948+0000 7fd11b297700 1 Processor -- start 2024-05-08T23:32:14.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 -- start start 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1140cdfa0 0x7fd114153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd1140ce380 0x7fd1141496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11414cc60 con 0x7fd1140c9ad0 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11414c5e0 con 0x7fd1140cdfa0 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.949+0000 7fd11b297700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11414bce0 con 0x7fd1140ce380 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48594/0 (socket says 172.21.15.33:48594) 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 -- 172.21.15.33:0/2759214023 learned_addr learned my addr 172.21.15.33:0/2759214023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119834700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd1140ce380 0x7fd1141496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd118832700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1140cdfa0 0x7fd114153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 -- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd1140ce380 msgr2=0x7fd1141496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd1140ce380 0x7fd1141496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 -- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1140cdfa0 msgr2=0x7fd114153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1140cdfa0 0x7fd114153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 -- 172.21.15.33:0/2759214023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040094a0 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd119033700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c550 tx=0x7fd11000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110004c80 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110004e00 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11414bfc0 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.950+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110007310 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.951+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11414c880 con 0x7fd1140c9ad0 2024-05-08T23:32:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.951+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11414cda0 con 0x7fd1140c9ad0 2024-05-08T23:32:14.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.952+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd110007470 con 0x7fd1140c9ad0 2024-05-08T23:32:14.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.952+0000 7fd109ffb700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 0x7fd0f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.953+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd110088fa0 con 0x7fd1140c9ad0 2024-05-08T23:32:14.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.953+0000 7fd118832700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 0x7fd0f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.953+0000 7fd118832700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 0x7fd0f806d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce790 tx=0x7fd104008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:14.957+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd11414cda0 con 0x7fd1140c9ad0 2024-05-08T23:32:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.177+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd11414cda0 con 0x7fd1140c9ad0 2024-05-08T23:32:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.179+0000 7fd109ffb700 1 -- 172.21.15.33:0/2759214023 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fd1100540e0 con 0x7fd1140c9ad0 2024-05-08T23:32:15.178 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:15.178 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 msgr2=0x7fd0f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 0x7fd0f806d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce790 tx=0x7fd104008030 comp rx=0 tx=0).stop 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 msgr2=0x7fd1141f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c550 tx=0x7fd11000c860 comp rx=0 tx=0).stop 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 shutdown_connections 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd0f806ac90 0x7fd0f806d150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd1140ce380 0x7fd1141496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1140c9ad0 0x7fd1141f30a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 --2- 172.21.15.33:0/2759214023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1140cdfa0 0x7fd114153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 >> 172.21.15.33:0/2759214023 conn(0x7fd114072230 msgr2=0x7fd114150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 shutdown_connections 2024-05-08T23:32:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.181+0000 7fd11b297700 1 -- 172.21.15.33:0/2759214023 wait complete. 2024-05-08T23:32:15.194 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:32:15.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.332+0000 7f97d8437700 1 Processor -- start 2024-05-08T23:32:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.332+0000 7f97d8437700 1 -- start start 2024-05-08T23:32:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d8437700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97d00c6e30 0x7f97d015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d8437700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d0154440 0x7f97d0070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d8437700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015e750 con 0x7f97d00c7210 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d8437700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d00709a0 con 0x7f97d00c6e30 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d8437700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015b150 con 0x7f97d0154440 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d61d3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97d00c6e30 0x7f97d015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d61d3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97d00c6e30 0x7f97d015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48606/0 (socket says 172.21.15.33:48606) 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d61d3700 1 -- 172.21.15.33:0/2383390564 learned_addr learned my addr 172.21.15.33:0/2383390564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.333+0000 7f97d69d4700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d0154440 0x7f97d0070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d61d3700 1 -- 172.21.15.33:0/2383390564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d0154440 msgr2=0x7f97d0070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d61d3700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d0154440 0x7f97d0070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d61d3700 1 -- 172.21.15.33:0/2383390564 >> v1:172.21.15.42:6789/0 conn(0x7f97d00c7210 legacy=0x7f97d0153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d61d3700 1 -- 172.21.15.33:0/2383390564 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d015dbd0 con 0x7f97d00c6e30 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d61d3700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c6e30 0x7f97d015ac10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009920 tx=0x7f97cc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f960468659f21c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d51d1700 1 -- 172.21.15.33:0/2383390564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc011830 con 0x7f97d00c6e30 2024-05-08T23:32:15.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d51d1700 1 -- 172.21.15.33:0/2383390564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc011990 con 0x7f97d00c6e30 2024-05-08T23:32:15.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.334+0000 7f97d51d1700 1 -- 172.21.15.33:0/2383390564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc011d20 con 0x7f97d00c6e30 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 -- 172.21.15.33:0/2383390564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c6e30 msgr2=0x7f97d015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c6e30 0x7f97d015ac10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009920 tx=0x7f97cc009c30 comp rx=0 tx=0).stop 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 -- 172.21.15.33:0/2383390564 shutdown_connections 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d0154440 0x7f97d0070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 --2- 172.21.15.33:0/2383390564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c6e30 0x7f97d015ac10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 -- 172.21.15.33:0/2383390564 >> 172.21.15.33:0/2383390564 conn(0x7f97d006a990 msgr2=0x7f97d006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 -- 172.21.15.33:0/2383390564 shutdown_connections 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.335+0000 7f97d8437700 1 -- 172.21.15.33:0/2383390564 wait complete. 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.337+0000 7f97d8437700 1 Processor -- start 2024-05-08T23:32:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.337+0000 7f97d8437700 1 -- start start 2024-05-08T23:32:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d00c6e30 0x7f97d0146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97d0154440 0x7f97d00c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01efcd0 con 0x7f97d00c7210 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01eff90 con 0x7f97d0154440 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d8437700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01f0270 con 0x7f97d00c6e30 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d59d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d59d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48618/0 (socket says 172.21.15.33:48618) 2024-05-08T23:32:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.338+0000 7f97d59d2700 1 -- 172.21.15.33:0/1561523922 learned_addr learned my addr 172.21.15.33:0/1561523922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:15.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 -- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d00c6e30 msgr2=0x7f97d0146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d00c6e30 0x7f97d0146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 -- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97d0154440 msgr2=0x7f97d00c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97d0154440 0x7f97d00c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 -- 172.21.15.33:0/1561523922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc009550 con 0x7f97d00c7210 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.339+0000 7f97d59d2700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c450 tx=0x7f97c000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.340+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00052d0 con 0x7f97d00c7210 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.340+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0005430 con 0x7f97d00c7210 2024-05-08T23:32:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.340+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00038c0 con 0x7f97d00c7210 2024-05-08T23:32:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.341+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d01f0550 con 0x7f97d00c7210 2024-05-08T23:32:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.341+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d01f0b10 con 0x7f97d00c7210 2024-05-08T23:32:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.342+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d00ca0c0 con 0x7f97d00c7210 2024-05-08T23:32:15.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.344+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f97c0003a20 con 0x7f97d00c7210 2024-05-08T23:32:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.345+0000 7f97c6ffd700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 0x7f97b406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.345+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97c0088ab0 con 0x7f97d00c7210 2024-05-08T23:32:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.345+0000 7f97d61d3700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 0x7f97b406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.345+0000 7f97d61d3700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 0x7f97b406d1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f97cc014040 tx=0x7f97cc010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.349+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f97c0053e30 con 0x7f97d00c7210 2024-05-08T23:32:15.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.562+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97d015e470 con 0x7f97b406ad10 2024-05-08T23:32:15.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.570+0000 7f97c6ffd700 1 -- 172.21.15.33:0/1561523922 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f97cc01e070 con 0x7f97b406ad10 2024-05-08T23:32:15.575 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 msgr2=0x7f97b406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 0x7f97b406d1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f97cc014040 tx=0x7f97cc010000 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 msgr2=0x7f97d00cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c450 tx=0x7f97c000c810 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 shutdown_connections 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97b406ad10 0x7f97b406d1d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97d0154440 0x7f97d00c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97d00c6e30 0x7f97d0146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 --2- 172.21.15.33:0/1561523922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97d00c7210 0x7f97d00cc780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 >> 172.21.15.33:0/1561523922 conn(0x7f97d006a990 msgr2=0x7f97d00c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 shutdown_connections 2024-05-08T23:32:15.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.574+0000 7f97d8437700 1 -- 172.21.15.33:0/1561523922 wait complete. 2024-05-08T23:32:15.579 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-08T23:32:14.771503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":2924544,"data_stored":1785288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907519"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679616+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:11.679616+0000","last_peered":"2024-05-08T23:32:11.679616+0000","last_clean":"2024-05-08T23:32:11.679616+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:11.679616+0000","last_undegraded":"2024-05-08T23:32:11.679616+0000","last_fullsized":"2024-05-08T23:32:11.679616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680727+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:11.680727+0000","last_peered":"2024-05-08T23:32:11.680727+0000","last_clean":"2024-05-08T23:32:11.680727+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:11.680727+0000","last_undegraded":"2024-05-08T23:32:11.680727+0000","last_fullsized":"2024-05-08T23:32:11.680727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680594+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:11.680594+0000","last_peered":"2024-05-08T23:32:11.680594+0000","last_clean":"2024-05-08T23:32:11.680594+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:11.680594+0000","last_undegraded":"2024-05-08T23:32:11.680594+0000","last_fullsized":"2024-05-08T23:32:11.680594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680096+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:11.680096+0000","last_peered":"2024-05-08T23:32:11.680096+0000","last_clean":"2024-05-08T23:32:11.680096+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:11.680096+0000","last_undegraded":"2024-05-08T23:32:11.680096+0000","last_fullsized":"2024-05-08T23:32:11.680096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680506+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:11.680506+0000","last_peered":"2024-05-08T23:32:11.680506+0000","last_clean":"2024-05-08T23:32:11.680506+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:11.680506+0000","last_undegraded":"2024-05-08T23:32:11.680506+0000","last_fullsized":"2024-05-08T23:32:11.680506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680188+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:11.680188+0000","last_peered":"2024-05-08T23:32:11.680188+0000","last_clean":"2024-05-08T23:32:11.680188+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:11.680188+0000","last_undegraded":"2024-05-08T23:32:11.680188+0000","last_fullsized":"2024-05-08T23:32:11.680188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680241+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:11.680241+0000","last_peered":"2024-05-08T23:32:11.680241+0000","last_clean":"2024-05-08T23:32:11.680241+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:11.680241+0000","last_undegraded":"2024-05-08T23:32:11.680241+0000","last_fullsized":"2024-05-08T23:32:11.680241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680056+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:11.680056+0000","last_peered":"2024-05-08T23:32:11.680056+0000","last_clean":"2024-05-08T23:32:11.680056+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:11.680056+0000","last_undegraded":"2024-05-08T23:32:11.680056+0000","last_fullsized":"2024-05-08T23:32:11.680056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679691+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:11.679691+0000","last_peered":"2024-05-08T23:32:11.679691+0000","last_clean":"2024-05-08T23:32:11.679691+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:11.679691+0000","last_undegraded":"2024-05-08T23:32:11.679691+0000","last_fullsized":"2024-05-08T23:32:11.679691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553408+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:11.553408+0000","last_peered":"2024-05-08T23:32:11.553408+0000","last_clean":"2024-05-08T23:32:11.553408+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:11.553408+0000","last_undegraded":"2024-05-08T23:32:11.553408+0000","last_fullsized":"2024-05-08T23:32:11.553408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340980+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:11.340980+0000","last_peered":"2024-05-08T23:32:11.340980+0000","last_clean":"2024-05-08T23:32:11.340980+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:11.340980+0000","last_undegraded":"2024-05-08T23:32:11.340980+0000","last_fullsized":"2024-05-08T23:32:11.340980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680064+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:11.680064+0000","last_peered":"2024-05-08T23:32:11.680064+0000","last_clean":"2024-05-08T23:32:11.680064+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:11.680064+0000","last_undegraded":"2024-05-08T23:32:11.680064+0000","last_fullsized":"2024-05-08T23:32:11.680064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680276+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:11.680276+0000","last_peered":"2024-05-08T23:32:11.680276+0000","last_clean":"2024-05-08T23:32:11.680276+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:11.680276+0000","last_undegraded":"2024-05-08T23:32:11.680276+0000","last_fullsized":"2024-05-08T23:32:11.680276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341178+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:11.341178+0000","last_peered":"2024-05-08T23:32:11.341178+0000","last_clean":"2024-05-08T23:32:11.341178+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:11.341178+0000","last_undegraded":"2024-05-08T23:32:11.341178+0000","last_fullsized":"2024-05-08T23:32:11.341178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679404+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:11.679404+0000","last_peered":"2024-05-08T23:32:11.679404+0000","last_clean":"2024-05-08T23:32:11.679404+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:11.679404+0000","last_undegraded":"2024-05-08T23:32:11.679404+0000","last_fullsized":"2024-05-08T23:32:11.679404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":811008,"data_stored":665937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":811008,"data_stored":665937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:15.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:15.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.766+0000 7f534bbd0700 1 Processor -- start 2024-05-08T23:32:15.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.766+0000 7f534bbd0700 1 -- start start 2024-05-08T23:32:15.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534bbd0700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f53440ceaa0 0x7f534415b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534bbd0700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440706d0 con 0x7f53441525e0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534bbd0700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440cb8c0 con 0x7f5344152200 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534bbd0700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53440cb300 con 0x7f53440ceaa0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534916b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f53441525e0 0x7f53440724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51920/0 (socket says 172.21.15.33:51920) 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534916b700 1 -- 172.21.15.33:0/1674986376 learned_addr learned my addr 172.21.15.33:0/1674986376 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534a16d700 1 --2- 172.21.15.33:0/1674986376 >> v2:172.21.15.42:3300/0 conn(0x7f53440ceaa0 0x7f534415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893024645 0 0) 0x7f53440706d0 con 0x7f53441525e0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.767+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53300035b0 con 0x7f53441525e0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3870453187 0 0) 0x7f53300035b0 con 0x7f53441525e0 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 >> v2:172.21.15.42:3300/0 conn(0x7f53440ceaa0 msgr2=0x7f534415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 --2- 172.21.15.33:0/1674986376 >> v2:172.21.15.42:3300/0 conn(0x7f53440ceaa0 0x7f534415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 >> v1:172.21.15.42:6789/0 conn(0x7f5344152200 legacy=0x7f53440caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440cb5e0 con 0x7f53441525e0 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f5334001670 con 0x7f53441525e0 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 >> v1:172.21.15.33:6789/0 conn(0x7f53441525e0 legacy=0x7f53440724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 0x7f5330003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330004850 0x7f5330004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5330005210 0x7f533000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53440cb300 con 0x7f5330005210 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533000c0b0 con 0x7f5330003a00 2024-05-08T23:32:15.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533000c270 con 0x7f5330004850 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534a16d700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5330005210 0x7f533000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 0x7f5330003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534916b700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330004850 0x7f5330004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 -- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330004850 msgr2=0x7f5330004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330004850 0x7f5330004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 -- 172.21.15.33:0/1674986376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5330005210 msgr2=0x7f533000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5330005210 0x7f533000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534996c700 1 -- 172.21.15.33:0/1674986376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53300035b0 con 0x7f5330003a00 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534a16d700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5330005210 0x7f533000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.768+0000 7f534a16d700 1 -- 172.21.15.33:0/1674986376 reap_dead start 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534996c700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 0x7f5330003e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53400017c0 tx=0x7f5330005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6615221852baa73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400052d0 con 0x7f5330003a00 2024-05-08T23:32:15.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340005430 con 0x7f5330003a00 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534896a700 1 -- 172.21.15.33:0/1674986376 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400038c0 con 0x7f5330003a00 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 -- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 msgr2=0x7f5330003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 0x7f5330003e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53400017c0 tx=0x7f5330005880 comp rx=0 tx=0).stop 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 -- 172.21.15.33:0/1674986376 shutdown_connections 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 --2- 172.21.15.33:0/1674986376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5330003a00 0x7f5330003e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 -- 172.21.15.33:0/1674986376 >> 172.21.15.33:0/1674986376 conn(0x7f534406a990 msgr2=0x7f534406e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 -- 172.21.15.33:0/1674986376 shutdown_connections 2024-05-08T23:32:15.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.769+0000 7f534bbd0700 1 -- 172.21.15.33:0/1674986376 wait complete. 2024-05-08T23:32:15.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.771+0000 7f534bbd0700 1 Processor -- start 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.771+0000 7f534bbd0700 1 -- start start 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 0x7f53441ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 0x7f5344148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441ee920 con 0x7f53441eedd0 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5344155b10 con 0x7f53440cd630 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534414c6c0 con 0x7f5330003a00 2024-05-08T23:32:15.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52738/0 (socket says 172.21.15.33:52738) 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 -- 172.21.15.33:0/277720589 learned_addr learned my addr 172.21.15.33:0/277720589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534996c700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 0x7f53441ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534a16d700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 0x7f5344148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 -- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 msgr2=0x7f53441ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 0x7f53441ee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 -- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 msgr2=0x7f5344148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 0x7f5344148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 -- 172.21.15.33:0/277720589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344157fc0 con 0x7f53441eedd0 2024-05-08T23:32:15.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534996c700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 0x7f53441ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534a16d700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 0x7f5344148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534916b700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53340072c0 tx=0x7f5334007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400f7d0 con 0x7f53441eedd0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400f970 con 0x7f53441eedd0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441582a0 con 0x7f53441eedd0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400fd20 con 0x7f53441eedd0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.772+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5344158860 con 0x7f53441eedd0 2024-05-08T23:32:15.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.773+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5344075460 con 0x7f53441eedd0 2024-05-08T23:32:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.774+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f533401d460 con 0x7f53441eedd0 2024-05-08T23:32:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.774+0000 7f533a7fc700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 0x7f532806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.775+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5334089360 con 0x7f53441eedd0 2024-05-08T23:32:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.775+0000 7f534996c700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 0x7f532806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.775+0000 7f534996c700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 0x7f532806d180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5340015040 tx=0x7f5340009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.779+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f53340544b0 con 0x7f53441eedd0 2024-05-08T23:32:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.987+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5344070360 con 0x7f532806acc0 2024-05-08T23:32:15.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.995+0000 7f533a7fc700 1 -- 172.21.15.33:0/277720589 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f5340021070 con 0x7f532806acc0 2024-05-08T23:32:15.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 msgr2=0x7f532806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 0x7f532806d180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5340015040 tx=0x7f5340009030 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 msgr2=0x7f534414be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53340072c0 tx=0x7f5334007680 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 shutdown_connections 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f532806acc0 0x7f532806d180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53440cd630 0x7f5344148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5330003a00 0x7f53441ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.998+0000 7f534bbd0700 1 --2- 172.21.15.33:0/277720589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53441eedd0 0x7f534414be50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.999+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 >> 172.21.15.33:0/277720589 conn(0x7f534406a990 msgr2=0x7f534406e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.999+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 shutdown_connections 2024-05-08T23:32:16.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:15.999+0000 7f534bbd0700 1 -- 172.21.15.33:0/277720589 wait complete. 2024-05-08T23:32:16.002 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:16.011 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-08T23:32:14.771503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219368,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755352,"statfs":{"total":773094113280,"available":772869480448,"internally_reserved":0,"allocated":2924544,"data_stored":1785288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.907519"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679616+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:11.679616+0000","last_peered":"2024-05-08T23:32:11.679616+0000","last_clean":"2024-05-08T23:32:11.679616+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:11.679616+0000","last_undegraded":"2024-05-08T23:32:11.679616+0000","last_fullsized":"2024-05-08T23:32:11.679616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680727+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:11.680727+0000","last_peered":"2024-05-08T23:32:11.680727+0000","last_clean":"2024-05-08T23:32:11.680727+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:11.680727+0000","last_undegraded":"2024-05-08T23:32:11.680727+0000","last_fullsized":"2024-05-08T23:32:11.680727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680594+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:11.680594+0000","last_peered":"2024-05-08T23:32:11.680594+0000","last_clean":"2024-05-08T23:32:11.680594+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:11.680594+0000","last_undegraded":"2024-05-08T23:32:11.680594+0000","last_fullsized":"2024-05-08T23:32:11.680594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680096+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:11.680096+0000","last_peered":"2024-05-08T23:32:11.680096+0000","last_clean":"2024-05-08T23:32:11.680096+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:11.680096+0000","last_undegraded":"2024-05-08T23:32:11.680096+0000","last_fullsized":"2024-05-08T23:32:11.680096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680506+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:11.680506+0000","last_peered":"2024-05-08T23:32:11.680506+0000","last_clean":"2024-05-08T23:32:11.680506+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:11.680506+0000","last_undegraded":"2024-05-08T23:32:11.680506+0000","last_fullsized":"2024-05-08T23:32:11.680506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680188+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:11.680188+0000","last_peered":"2024-05-08T23:32:11.680188+0000","last_clean":"2024-05-08T23:32:11.680188+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:11.680188+0000","last_undegraded":"2024-05-08T23:32:11.680188+0000","last_fullsized":"2024-05-08T23:32:11.680188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680241+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:11.680241+0000","last_peered":"2024-05-08T23:32:11.680241+0000","last_clean":"2024-05-08T23:32:11.680241+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:11.680241+0000","last_undegraded":"2024-05-08T23:32:11.680241+0000","last_fullsized":"2024-05-08T23:32:11.680241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680056+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:11.680056+0000","last_peered":"2024-05-08T23:32:11.680056+0000","last_clean":"2024-05-08T23:32:11.680056+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:11.680056+0000","last_undegraded":"2024-05-08T23:32:11.680056+0000","last_fullsized":"2024-05-08T23:32:11.680056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679691+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:11.679691+0000","last_peered":"2024-05-08T23:32:11.679691+0000","last_clean":"2024-05-08T23:32:11.679691+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:11.679691+0000","last_undegraded":"2024-05-08T23:32:11.679691+0000","last_fullsized":"2024-05-08T23:32:11.679691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553408+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:11.553408+0000","last_peered":"2024-05-08T23:32:11.553408+0000","last_clean":"2024-05-08T23:32:11.553408+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:11.553408+0000","last_undegraded":"2024-05-08T23:32:11.553408+0000","last_fullsized":"2024-05-08T23:32:11.553408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340980+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:11.340980+0000","last_peered":"2024-05-08T23:32:11.340980+0000","last_clean":"2024-05-08T23:32:11.340980+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:11.340980+0000","last_undegraded":"2024-05-08T23:32:11.340980+0000","last_fullsized":"2024-05-08T23:32:11.340980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680064+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:11.680064+0000","last_peered":"2024-05-08T23:32:11.680064+0000","last_clean":"2024-05-08T23:32:11.680064+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:11.680064+0000","last_undegraded":"2024-05-08T23:32:11.680064+0000","last_fullsized":"2024-05-08T23:32:11.680064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680276+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:11.680276+0000","last_peered":"2024-05-08T23:32:11.680276+0000","last_clean":"2024-05-08T23:32:11.680276+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:11.680276+0000","last_undegraded":"2024-05-08T23:32:11.680276+0000","last_fullsized":"2024-05-08T23:32:11.680276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341178+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:11.341178+0000","last_peered":"2024-05-08T23:32:11.341178+0000","last_clean":"2024-05-08T23:32:11.341178+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:11.341178+0000","last_undegraded":"2024-05-08T23:32:11.341178+0000","last_fullsized":"2024-05-08T23:32:11.341178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679404+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:11.679404+0000","last_peered":"2024-05-08T23:32:11.679404+0000","last_clean":"2024-05-08T23:32:11.679404+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:11.679404+0000","last_undegraded":"2024-05-08T23:32:11.679404+0000","last_fullsized":"2024-05-08T23:32:11.679404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":811008,"data_stored":665937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":811008,"data_stored":665937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:16.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:32:16.016 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:32:16.016 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:32:16.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T23:32:16.017 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T23:32:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.191+0000 7f24c87df700 1 Processor -- start 2024-05-08T23:32:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.191+0000 7f24c87df700 1 -- start start 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.192+0000 7f24c87df700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f24c0151410 0x7f24c00c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.192+0000 7f24c87df700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00c8610 con 0x7f24c00c8ea0 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.192+0000 7f24c87df700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00c6cf0 con 0x7f24c00c9280 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.192+0000 7f24c87df700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c00c4470 con 0x7f24c0151410 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c657b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f24c00c8ea0 0x7f24c0071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51930/0 (socket says 172.21.15.33:51930) 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c657b700 1 -- 172.21.15.33:0/2378864277 learned_addr learned my addr 172.21.15.33:0/2378864277 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c6d7c700 1 --2- 172.21.15.33:0/2378864277 >> v2:172.21.15.42:3300/0 conn(0x7f24c0151410 0x7f24c00c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c5579700 1 -- 172.21.15.33:0/2378864277 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010605576 0 0) 0x7f24c00c8610 con 0x7f24c00c8ea0 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c5579700 1 -- 172.21.15.33:0/2378864277 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24ac0035b0 con 0x7f24c00c8ea0 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2378864277 >> v1:172.21.15.42:6789/0 conn(0x7f24c00c9280 legacy=0x7f24c0150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2378864277 >> v1:172.21.15.33:6789/0 conn(0x7f24c00c8ea0 legacy=0x7f24c0071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2378864277 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00c6fd0 con 0x7f24c0151410 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c6d7c700 1 --2- 172.21.15.33:0/2378864277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c0151410 0x7f24c00c7f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009d90 tx=0x7f24b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fca9846ac7c6dd3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c5579700 1 -- 172.21.15.33:0/2378864277 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800e070 con 0x7f24c0151410 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c5579700 1 -- 172.21.15.33:0/2378864277 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800bb20 con 0x7f24c0151410 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.193+0000 7f24c5579700 1 -- 172.21.15.33:0/2378864277 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012430 con 0x7f24c0151410 2024-05-08T23:32:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 -- 172.21.15.33:0/2378864277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c0151410 msgr2=0x7f24c00c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 --2- 172.21.15.33:0/2378864277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c0151410 0x7f24c00c7f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009d90 tx=0x7f24b800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 -- 172.21.15.33:0/2378864277 shutdown_connections 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 --2- 172.21.15.33:0/2378864277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c0151410 0x7f24c00c7f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 -- 172.21.15.33:0/2378864277 >> 172.21.15.33:0/2378864277 conn(0x7f24c006a990 msgr2=0x7f24c006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 -- 172.21.15.33:0/2378864277 shutdown_connections 2024-05-08T23:32:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.194+0000 7f24c87df700 1 -- 172.21.15.33:0/2378864277 wait complete. 2024-05-08T23:32:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.196+0000 7f24c87df700 1 Processor -- start 2024-05-08T23:32:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.196+0000 7f24c87df700 1 -- start start 2024-05-08T23:32:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c00c8ea0 0x7f24c01ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24c00c9280 0x7f24c01f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0156380 con 0x7f24c0151410 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c0146790 con 0x7f24c00c8ea0 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c87df700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01f2100 con 0x7f24c00c9280 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52746/0 (socket says 172.21.15.33:52746) 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2034113352 learned_addr learned my addr 172.21.15.33:0/2034113352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c657b700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c00c8ea0 0x7f24c01ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c5d7a700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24c00c9280 0x7f24c01f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24c00c9280 msgr2=0x7f24c01f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24c00c9280 0x7f24c01f1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c00c8ea0 msgr2=0x7f24c01ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c00c8ea0 0x7f24c01ef2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.197+0000 7f24c6d7c700 1 -- 172.21.15.33:0/2034113352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009a70 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24c6d7c700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b680 tx=0x7f24b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012430 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8020070 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c01565d0 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8004280 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.198+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c015aa60 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.199+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c00c9980 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.199+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f24b80043e0 con 0x7f24c0151410 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.200+0000 7f24b6ffd700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 0x7f24a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.200+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b8022070 con 0x7f24c0151410 2024-05-08T23:32:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.200+0000 7f24c657b700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 0x7f24a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.200+0000 7f24c657b700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 0x7f24a406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.204+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f24b8056140 con 0x7f24c0151410 2024-05-08T23:32:16.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.492+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f24c0075460 con 0x7f24c0151410 2024-05-08T23:32:16.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.496+0000 7f24b6ffd700 1 -- 172.21.15.33:0/2034113352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170936 (secure 0 0 0) 0x7f24b8059760 con 0x7f24c0151410 2024-05-08T23:32:16.497 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:16.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.498+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 msgr2=0x7f24a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.498+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 0x7f24a406d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0007dc0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.498+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 msgr2=0x7f24c0143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.498+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24b800b680 tx=0x7f24b8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 shutdown_connections 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f24a406ac90 0x7f24a406d150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24c0151410 0x7f24c0143520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24c00c8ea0 0x7f24c01ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 --2- 172.21.15.33:0/2034113352 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24c00c9280 0x7f24c01f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 >> 172.21.15.33:0/2034113352 conn(0x7f24c006a990 msgr2=0x7f24c014f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 shutdown_connections 2024-05-08T23:32:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.499+0000 7f24c87df700 1 -- 172.21.15.33:0/2034113352 wait complete. 2024-05-08T23:32:16.511 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":46840},{"type":"v1","addr":"172.21.15.33:6833","nonce":46840}]},"active_addr":"172.21.15.33:6833/46840","active_change":"2024-05-08T23:31:58.762723+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":474261686}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1119360525}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3846468262}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3176618545}]}]} 2024-05-08T23:32:16.515 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T23:32:16.515 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T23:32:16.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:16.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.652+0000 7fcece3e8700 1 Processor -- start 2024-05-08T23:32:16.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.652+0000 7fcece3e8700 1 -- start start 2024-05-08T23:32:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.652+0000 7fcece3e8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.652+0000 7fcece3e8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcec8071700 0x7fcec815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.652+0000 7fcece3e8700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80c79f0 con 0x7fcec815bb30 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcece3e8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec8158a90 con 0x7fcec8071700 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcece3e8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec806faa0 con 0x7fcec8071320 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcec8071700 0x7fcec815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcec8071700 0x7fcec815b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52756/0 (socket says 172.21.15.33:52756) 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 -- 172.21.15.33:0/3357381789 learned_addr learned my addr 172.21.15.33:0/3357381789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec7fff700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 -- 172.21.15.33:0/3357381789 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 msgr2=0x7fcec806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec806b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 -- 172.21.15.33:0/3357381789 >> v1:172.21.15.42:6789/0 conn(0x7fcec815bb30 legacy=0x7fcec8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 -- 172.21.15.33:0/3357381789 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec806b8b0 con 0x7fcec8071700 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec77fe700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec815b5f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0098f0 tx=0x7fcebc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d6699c3005f8e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec6ffd700 1 -- 172.21.15.33:0/3357381789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00b8a0 con 0x7fcec8071700 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec6ffd700 1 -- 172.21.15.33:0/3357381789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00ba00 con 0x7fcec8071700 2024-05-08T23:32:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.653+0000 7fcec6ffd700 1 -- 172.21.15.33:0/3357381789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00bd90 con 0x7fcec8071700 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 -- 172.21.15.33:0/3357381789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 msgr2=0x7fcec815b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec815b5f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0098f0 tx=0x7fcebc009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 -- 172.21.15.33:0/3357381789 shutdown_connections 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 --2- 172.21.15.33:0/3357381789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec815b5f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 -- 172.21.15.33:0/3357381789 >> 172.21.15.33:0/3357381789 conn(0x7fcec806e990 msgr2=0x7fcec806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 -- 172.21.15.33:0/3357381789 shutdown_connections 2024-05-08T23:32:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.654+0000 7fcece3e8700 1 -- 172.21.15.33:0/3357381789 wait complete. 2024-05-08T23:32:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.656+0000 7fcece3e8700 1 Processor -- start 2024-05-08T23:32:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.656+0000 7fcece3e8700 1 -- start start 2024-05-08T23:32:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec815bb30 0x7fcec81edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec81e1f90 con 0x7fcec8071700 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec81e3500 con 0x7fcec815bb30 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcece3e8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec815ae70 con 0x7fcec8071320 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52768/0 (socket says 172.21.15.33:52768) 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 -- 172.21.15.33:0/2446371787 learned_addr learned my addr 172.21.15.33:0/2446371787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:16.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec7fff700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcecc985700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec815bb30 0x7fcec81edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 -- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 msgr2=0x7fcec815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 -- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec815bb30 msgr2=0x7fcec81edc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec815bb30 0x7fcec81edc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec77fe700 1 -- 172.21.15.33:0/2446371787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009550 con 0x7fcec8071700 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.657+0000 7fcec7fff700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:16.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcec77fe700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009c40 tx=0x7fcebc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc004480 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc01f070 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec815b350 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0038c0 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.658+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec81e15f0 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.659+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec81e20d0 con 0x7fcec8071700 2024-05-08T23:32:16.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.659+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcebc021030 con 0x7fcec8071700 2024-05-08T23:32:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.660+0000 7fcec4ff9700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 0x7fceb406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.660+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcebc08a100 con 0x7fcec8071700 2024-05-08T23:32:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.660+0000 7fcec7fff700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 0x7fceb406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.660+0000 7fcec7fff700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 0x7fceb406d180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a00 tx=0x7fceb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:16.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.665+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcec81e20d0 con 0x7fcec8071700 2024-05-08T23:32:16.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.887+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcec81e20d0 con 0x7fcec8071700 2024-05-08T23:32:16.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.889+0000 7fcec4ff9700 1 -- 172.21.15.33:0/2446371787 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fcec81e20d0 con 0x7fcec8071700 2024-05-08T23:32:16.888 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:16.888 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:16.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 msgr2=0x7fceb406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 0x7fceb406d180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009a00 tx=0x7fceb8008040 comp rx=0 tx=0).stop 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 msgr2=0x7fcec81e4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009c40 tx=0x7fcebc004750 comp rx=0 tx=0).stop 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 shutdown_connections 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fceb406acc0 0x7fceb406d180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec815bb30 0x7fcec81edc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcec8071320 0x7fcec815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 --2- 172.21.15.33:0/2446371787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec8071700 0x7fcec81e4bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:16.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 >> 172.21.15.33:0/2446371787 conn(0x7fcec806e990 msgr2=0x7fcec806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:16.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 shutdown_connections 2024-05-08T23:32:16.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:16.891+0000 7fcece3e8700 1 -- 172.21.15.33:0/2446371787 wait complete. 2024-05-08T23:32:16.904 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T23:32:16.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.041+0000 7fa00354a700 1 Processor -- start 2024-05-08T23:32:17.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 -- start start 2024-05-08T23:32:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9ffc0cdfa0 0x7f9ffc0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ffc0ce380 0x7f9ffc0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ffc15b140 con 0x7f9ffc153ef0 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0c8f30 con 0x7f9ffc0ce380 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.042+0000 7fa00354a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc15a5c0 con 0x7f9ffc0cdfa0 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ffc0ce380 0x7f9ffc0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9ffc0ce380 0x7f9ffc0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52772/0 (socket says 172.21.15.33:52772) 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 -- 172.21.15.33:0/1242121209 learned_addr learned my addr 172.21.15.33:0/1242121209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa0012e6700 1 --2- 172.21.15.33:0/1242121209 >> v2:172.21.15.42:3300/0 conn(0x7f9ffc0cdfa0 0x7f9ffc0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 -- 172.21.15.33:0/1242121209 >> v2:172.21.15.42:3300/0 conn(0x7f9ffc0cdfa0 msgr2=0x7f9ffc0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 --2- 172.21.15.33:0/1242121209 >> v2:172.21.15.42:3300/0 conn(0x7f9ffc0cdfa0 0x7f9ffc0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 -- 172.21.15.33:0/1242121209 >> v1:172.21.15.42:6789/0 conn(0x7f9ffc153ef0 legacy=0x7f9ffc154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 -- 172.21.15.33:0/1242121209 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc15a8a0 con 0x7f9ffc0ce380 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7fa000ae5700 1 --2- 172.21.15.33:0/1242121209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc0ce380 0x7f9ffc0c9900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc069230 tx=0x7f9fec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48550a856cf8abe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7f9ff3fff700 1 -- 172.21.15.33:0/1242121209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00b870 con 0x7f9ffc0ce380 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7f9ff3fff700 1 -- 172.21.15.33:0/1242121209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00b9d0 con 0x7f9ffc0ce380 2024-05-08T23:32:17.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.043+0000 7f9ff3fff700 1 -- 172.21.15.33:0/1242121209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00bd60 con 0x7f9ffc0ce380 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 -- 172.21.15.33:0/1242121209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc0ce380 msgr2=0x7f9ffc0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 --2- 172.21.15.33:0/1242121209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc0ce380 0x7f9ffc0c9900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc069230 tx=0x7f9fec009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 -- 172.21.15.33:0/1242121209 shutdown_connections 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 --2- 172.21.15.33:0/1242121209 >> v2:172.21.15.42:3300/0 conn(0x7f9ffc0cdfa0 0x7f9ffc0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 --2- 172.21.15.33:0/1242121209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc0ce380 0x7f9ffc0c9900 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 -- 172.21.15.33:0/1242121209 >> 172.21.15.33:0/1242121209 conn(0x7f9ffc072230 msgr2=0x7f9ffc072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 -- 172.21.15.33:0/1242121209 shutdown_connections 2024-05-08T23:32:17.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.044+0000 7fa00354a700 1 -- 172.21.15.33:0/1242121209 wait complete. 2024-05-08T23:32:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.045+0000 7fa00354a700 1 Processor -- start 2024-05-08T23:32:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.045+0000 7fa00354a700 1 -- start start 2024-05-08T23:32:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 0x7f9ffc1465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ffc0ce380 0x7f9ffc148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc14d540 con 0x7f9ffc153ef0 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc1491e0 con 0x7f9ffc0cdfa0 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa00354a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc1f3bf0 con 0x7f9ffc0ce380 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52780/0 (socket says 172.21.15.33:52780) 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 -- 172.21.15.33:0/3765085803 learned_addr learned my addr 172.21.15.33:0/3765085803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa0012e6700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 0x7f9ffc1465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa000ae5700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ffc0ce380 0x7f9ffc148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 -- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ffc0ce380 msgr2=0x7f9ffc148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ffc0ce380 0x7f9ffc148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 -- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 msgr2=0x7f9ffc1465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 0x7f9ffc1465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa001ae7700 1 -- 172.21.15.33:0/3765085803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009550 con 0x7f9ffc153ef0 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.046+0000 7fa0012e6700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 0x7f9ffc1465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:17.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7fa001ae7700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400cc50 tx=0x7f9ff400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400ad80 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff400ce50 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc14a460 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4003c40 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.047+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc14aa20 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.048+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc075460 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.048+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9ff40050d0 con 0x7f9ffc153ef0 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.049+0000 7f9ff1ffb700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 0x7f9fe006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.049+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ff4089460 con 0x7f9ffc153ef0 2024-05-08T23:32:17.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.049+0000 7fa0012e6700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 0x7f9fe006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.049+0000 7fa0012e6700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 0x7f9fe006d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009820 tx=0x7f9ff8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.054+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9ff4054630 con 0x7f9ffc153ef0 2024-05-08T23:32:17.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:17.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.275+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ffc0c9cd0 con 0x7f9ffc153ef0 2024-05-08T23:32:17.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.277+0000 7f9ff1ffb700 1 -- 172.21.15.33:0/3765085803 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f9ff4057c50 con 0x7f9ffc153ef0 2024-05-08T23:32:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 msgr2=0x7f9fe006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 0x7f9fe006d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009820 tx=0x7f9ff8008040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 msgr2=0x7f9ffc14e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400cc50 tx=0x7f9ff400a340 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 shutdown_connections 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9fe006ac90 0x7f9fe006d150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ffc153ef0 0x7f9ffc14e9f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ffc0cdfa0 0x7f9ffc1465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 --2- 172.21.15.33:0/3765085803 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ffc0ce380 0x7f9ffc148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 >> 172.21.15.33:0/3765085803 conn(0x7f9ffc072230 msgr2=0x7f9ffc0c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 shutdown_connections 2024-05-08T23:32:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.279+0000 7fa00354a700 1 -- 172.21.15.33:0/3765085803 wait complete. 2024-05-08T23:32:17.292 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T23:32:17.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T23:32:17.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T23:32:17.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T23:32:17.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T23:32:17.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T23:32:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T23:32:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T23:32:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.538+0000 7ff9d1901700 1 Processor -- start 2024-05-08T23:32:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.538+0000 7ff9d1901700 1 -- start start 2024-05-08T23:32:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7fefe7194700 1 Processor -- start 2024-05-08T23:32:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7fefe7194700 1 -- start start 2024-05-08T23:32:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7f952c778700 1 Processor -- start 2024-05-08T23:32:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7ff9d1901700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9cc0c9050 0x7ff9cc070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7ff9d1901700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0713d0 0x7ff9cc070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7ff9d1901700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc072e40 con 0x7ff9cc0c3020 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7ff9d1901700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc06b920 con 0x7ff9cc0c9050 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7ff9d1901700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc06aa30 con 0x7ff9cc0713d0 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.538+0000 7f748c558700 1 Processor -- start 2024-05-08T23:32:17.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.538+0000 7f748c558700 1 -- start start 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fefe0156f50 0x7fefe0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefe01504a0 0x7fefe0150920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe0071790 con 0x7fefe01504a0 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe0072880 con 0x7fefe0156f50 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe7194700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe006b550 con 0x7fefe00c6260 2024-05-08T23:32:17.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f748c558700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f74840c2770 0x7f74840c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f748c558700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74840c7d80 0x7f7484070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f748c558700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748406ba40 con 0x7f74841589d0 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f748c558700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748415f180 con 0x7f74840c7d80 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f748c558700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840c8210 con 0x7f74840c2770 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fefe4f30700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53504/0 (socket says 172.21.15.33:53504) 2024-05-08T23:32:17.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 -- 172.21.15.33:0/3427734477 learned_addr learned my addr 172.21.15.33:0/3427734477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7f952c778700 1 -- start start 2024-05-08T23:32:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.539+0000 7fe19b2af700 1 Processor -- start 2024-05-08T23:32:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7fe19b2af700 1 -- start start 2024-05-08T23:32:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe5731700 1 --2- 172.21.15.33:0/3427734477 >> v2:172.21.15.33:3300/0 conn(0x7fefe01504a0 0x7fefe0150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f748a2f4700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f74841589d0 0x7f748406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51946/0 (socket says 172.21.15.33:51946) 2024-05-08T23:32:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f748a2f4700 1 -- 172.21.15.33:0/3158880458 learned_addr learned my addr 172.21.15.33:0/3158880458 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f7489af3700 1 --2- 172.21.15.33:0/3158880458 >> v2:172.21.15.42:3300/0 conn(0x7f74840c2770 0x7f74840c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f748aaf5700 1 --2- 172.21.15.33:0/3158880458 >> v2:172.21.15.33:3300/0 conn(0x7f74840c7d80 0x7f7484070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 -- 172.21.15.33:0/3427734477 >> v2:172.21.15.42:3300/0 conn(0x7fefe0156f50 msgr2=0x7fefe0071bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 --2- 172.21.15.33:0/3427734477 >> v2:172.21.15.42:3300/0 conn(0x7fefe0156f50 0x7fefe0071bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 -- 172.21.15.33:0/3427734477 >> v2:172.21.15.33:3300/0 conn(0x7fefe01504a0 msgr2=0x7fefe0150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 --2- 172.21.15.33:0/3427734477 >> v2:172.21.15.33:3300/0 conn(0x7fefe01504a0 0x7fefe0150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 -- 172.21.15.33:0/3427734477 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe006b270 con 0x7fefe00c6260 2024-05-08T23:32:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9cc0c9050 0x7ff9cc070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9cc0c9050 0x7ff9cc070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54806/0 (socket says 172.21.15.33:54806) 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 -- 172.21.15.33:0/3191468356 learned_addr learned my addr 172.21.15.33:0/3191468356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9cb7fe700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0713d0 0x7ff9cc070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 -- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0713d0 msgr2=0x7ff9cc070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0713d0 0x7ff9cc070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.540+0000 7ff9caffd700 1 -- 172.21.15.33:0/3191468356 >> v1:172.21.15.42:6789/0 conn(0x7ff9cc0c3020 legacy=0x7ff9cc070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7ff9caffd700 1 -- 172.21.15.33:0/3191468356 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc06ad10 con 0x7ff9cc0c9050 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7ff9caffd700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0c9050 0x7ff9cc070010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098f0 tx=0x7ff9c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9721c6e5fbf8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f952c778700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f952c778700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 0x7f95241577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f952c778700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95240c8110 con 0x7f9524156db0 2024-05-08T23:32:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f952c778700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95240c7e30 con 0x7f95240c70c0 2024-05-08T23:32:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f952c778700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95240c6300 con 0x7f95240c8e50 2024-05-08T23:32:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fefe4f30700 1 --2- 172.21.15.33:0/3427734477 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefe006cf50 tx=0x7fefd800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed40a43728bb6265 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7ff9c9ffb700 1 -- 172.21.15.33:0/3191468356 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004230 con 0x7ff9cc0c9050 2024-05-08T23:32:17.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f7489af3700 1 -- 172.21.15.33:0/3158880458 >> v2:172.21.15.33:3300/0 conn(0x7f74840c7d80 msgr2=0x7f7484070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f7489af3700 1 --2- 172.21.15.33:0/3158880458 >> v2:172.21.15.33:3300/0 conn(0x7f74840c7d80 0x7f7484070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f7489af3700 1 -- 172.21.15.33:0/3158880458 >> v1:172.21.15.33:6789/0 conn(0x7f74841589d0 legacy=0x7f748406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7f7489af3700 1 -- 172.21.15.33:0/3158880458 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484070930 con 0x7f74840c2770 2024-05-08T23:32:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7ff9c9ffb700 1 -- 172.21.15.33:0/3191468356 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c0004390 con 0x7ff9cc0c9050 2024-05-08T23:32:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7ff9c9ffb700 1 -- 172.21.15.33:0/3191468356 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00036a0 con 0x7ff9cc0c9050 2024-05-08T23:32:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53506/0 (socket says 172.21.15.33:53506) 2024-05-08T23:32:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 -- 172.21.15.33:0/2020650938 learned_addr learned my addr 172.21.15.33:0/2020650938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fe19b2af700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe1941589d0 0x7fe1940c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fe19b2af700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe194070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fe19b2af700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19406ba40 con 0x7fe1940c2770 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fe19b2af700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19415f180 con 0x7fe1941589d0 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.541+0000 7fe19b2af700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940c8210 con 0x7fe1940c7d80 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19984c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe194070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe1941589d0 0x7fe1940c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19984c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe194070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53516/0 (socket says 172.21.15.33:53516) 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19984c700 1 -- 172.21.15.33:0/1727913595 learned_addr learned my addr 172.21.15.33:0/1727913595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f9529d13700 1 --2- 172.21.15.33:0/2020650938 >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 0x7f95241577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 -- 172.21.15.33:0/2020650938 >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 msgr2=0x7f95241577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 --2- 172.21.15.33:0/2020650938 >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 0x7f95241577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 -- 172.21.15.33:0/2020650938 >> v1:172.21.15.33:6789/0 conn(0x7f9524156db0 legacy=0x7f9524150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 -- 172.21.15.33:0/2020650938 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240c65e0 con 0x7f95240c8e50 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fefdf7fe700 1 -- 172.21.15.33:0/3427734477 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800e070 con 0x7fefe00c6260 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fefdf7fe700 1 -- 172.21.15.33:0/3427734477 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800b980 con 0x7fefe00c6260 2024-05-08T23:32:17.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fefdf7fe700 1 -- 172.21.15.33:0/3427734477 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800bd30 con 0x7fefe00c6260 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f7489af3700 1 --2- 172.21.15.33:0/3158880458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74840c2770 0x7f74840c21a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748415c380 tx=0x7f7474009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c92f63df67d46e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f9529d13700 1 --2- 172.21.15.33:0/2020650938 >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 0x7f95241577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7f952a514700 1 --2- 172.21.15.33:0/2020650938 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f951c009e50 tx=0x7f951c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a46ef09e4d49a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 -- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 msgr2=0x7fe194070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 --2- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe194070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 -- 172.21.15.33:0/1727913595 >> v1:172.21.15.33:6789/0 conn(0x7fe1940c2770 legacy=0x7fe19406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 -- 172.21.15.33:0/1727913595 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194070930 con 0x7fe1941589d0 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.542+0000 7fe19904b700 1 --2- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe1940c21a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe190009840 tx=0x7fe190009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dc8461d17dcd1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fe18bfff700 1 -- 172.21.15.33:0/1727913595 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190004230 con 0x7fe1941589d0 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 -- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0c9050 msgr2=0x7ff9cc070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0c9050 0x7ff9cc070010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00098f0 tx=0x7ff9c0009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 -- 172.21.15.33:0/3191468356 shutdown_connections 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0713d0 0x7ff9cc070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 --2- 172.21.15.33:0/3191468356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0c9050 0x7ff9cc070010 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 -- 172.21.15.33:0/3191468356 >> 172.21.15.33:0/3191468356 conn(0x7ff9cc0c8360 msgr2=0x7ff9cc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fefe7194700 1 -- 172.21.15.33:0/3427734477 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 msgr2=0x7fefe006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fefe7194700 1 --2- 172.21.15.33:0/3427734477 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fefe006cf50 tx=0x7fefd800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 -- 172.21.15.33:0/3191468356 shutdown_connections 2024-05-08T23:32:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7ff9d1901700 1 -- 172.21.15.33:0/3191468356 wait complete. 2024-05-08T23:32:17.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f74892f2700 1 -- 172.21.15.33:0/3158880458 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474011830 con 0x7f74840c2770 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f74892f2700 1 -- 172.21.15.33:0/3158880458 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474011990 con 0x7f74840c2770 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f74892f2700 1 -- 172.21.15.33:0/3158880458 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474011d20 con 0x7f74840c2770 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fe18bfff700 1 -- 172.21.15.33:0/1727913595 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe190004390 con 0x7fe1941589d0 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fe18bfff700 1 -- 172.21.15.33:0/1727913595 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900036a0 con 0x7fe1941589d0 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fead725f700 1 Processor -- start 2024-05-08T23:32:17.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7fead725f700 1 -- start start 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f9529512700 1 -- 172.21.15.33:0/2020650938 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00e070 con 0x7f95240c8e50 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f9529512700 1 -- 172.21.15.33:0/2020650938 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c003a20 con 0x7f95240c8e50 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.543+0000 7f9529512700 1 -- 172.21.15.33:0/2020650938 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c003db0 con 0x7f95240c8e50 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 -- 172.21.15.33:0/2020650938 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 msgr2=0x7f95240c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 --2- 172.21.15.33:0/2020650938 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f951c009e50 tx=0x7f951c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fe19b2af700 1 -- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 msgr2=0x7fe1940c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fe19b2af700 1 --2- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe1940c21a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe190009840 tx=0x7fe190009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 -- 172.21.15.33:0/2020650938 shutdown_connections 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 --2- 172.21.15.33:0/2020650938 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c8e50 0x7f95240c7780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 --2- 172.21.15.33:0/2020650938 >> v2:172.21.15.42:3300/0 conn(0x7f95240c70c0 0x7f95241577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 -- 172.21.15.33:0/2020650938 >> 172.21.15.33:0/2020650938 conn(0x7f95240c8360 msgr2=0x7f95240c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 -- 172.21.15.33:0/3427734477 shutdown_connections 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 --2- 172.21.15.33:0/3427734477 >> v2:172.21.15.33:3300/0 conn(0x7fefe01504a0 0x7fefe0150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 --2- 172.21.15.33:0/3427734477 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe00c6260 0x7fefe006a990 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 --2- 172.21.15.33:0/3427734477 >> v2:172.21.15.42:3300/0 conn(0x7fefe0156f50 0x7fefe0071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 -- 172.21.15.33:0/3427734477 >> 172.21.15.33:0/3427734477 conn(0x7fefe00c8360 msgr2=0x7fefe00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7f952c778700 1 -- 172.21.15.33:0/2020650938 shutdown_connections 2024-05-08T23:32:17.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.544+0000 7fefe7194700 1 -- 172.21.15.33:0/3427734477 shutdown_connections 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 -- 172.21.15.33:0/3158880458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74840c2770 msgr2=0x7f74840c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 --2- 172.21.15.33:0/3158880458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74840c2770 0x7f74840c21a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f748415c380 tx=0x7f7474009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 -- 172.21.15.33:0/3158880458 shutdown_connections 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 --2- 172.21.15.33:0/3158880458 >> v2:172.21.15.33:3300/0 conn(0x7f74840c7d80 0x7f7484070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 --2- 172.21.15.33:0/3158880458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74840c2770 0x7f74840c21a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f748c558700 1 -- 172.21.15.33:0/3158880458 >> 172.21.15.33:0/3158880458 conn(0x7f74840c8360 msgr2=0x7f74840c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead725f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead725f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fead00b3200 0x7fead00cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead725f700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead00cb2f0 con 0x7fead006c240 2024-05-08T23:32:17.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead725f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead00cb6c0 con 0x7fead00b3200 2024-05-08T23:32:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead725f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead00cb940 con 0x7fead015f460 2024-05-08T23:32:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7ff9d1901700 1 Processor -- start 2024-05-08T23:32:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fefe7194700 1 -- 172.21.15.33:0/3427734477 wait complete. 2024-05-08T23:32:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7f748c558700 1 -- 172.21.15.33:0/3158880458 shutdown_connections 2024-05-08T23:32:17.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7feacffff700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fead006c240 0x7fead006b400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59008/0 (socket says 172.21.15.33:59008) 2024-05-08T23:32:17.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7feacffff700 1 -- 172.21.15.33:0/1332854157 learned_addr learned my addr 172.21.15.33:0/1332854157 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead4ffb700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7fead57fc700 1 --2- 172.21.15.33:0/1332854157 >> v2:172.21.15.33:3300/0 conn(0x7fead00b3200 0x7fead00cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.545+0000 7f952c778700 1 -- 172.21.15.33:0/2020650938 wait complete. 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7f748c558700 1 -- 172.21.15.33:0/3158880458 wait complete. 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7feacf7fe700 1 -- 172.21.15.33:0/1332854157 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448400337 0 0) 0x7fead00cb2f0 con 0x7fead006c240 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7feacf7fe700 1 -- 172.21.15.33:0/1332854157 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab80035b0 con 0x7fead006c240 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7fead57fc700 1 -- 172.21.15.33:0/1332854157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 msgr2=0x7fead00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7fead57fc700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7fead57fc700 1 -- 172.21.15.33:0/1332854157 >> v1:172.21.15.42:6789/0 conn(0x7fead006c240 legacy=0x7fead006b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7fead57fc700 1 -- 172.21.15.33:0/1332854157 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead00cbc20 con 0x7fead00b3200 2024-05-08T23:32:17.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7f952c778700 1 Processor -- start 2024-05-08T23:32:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7fefe7194700 1 Processor -- start 2024-05-08T23:32:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fead57fc700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead00b3200 0x7fead00cd070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feac0009840 tx=0x7feac0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c64037762e0c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7feacf7fe700 1 -- 172.21.15.33:0/1332854157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0004230 con 0x7fead00b3200 2024-05-08T23:32:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7feacf7fe700 1 -- 172.21.15.33:0/1332854157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac0004390 con 0x7fead00b3200 2024-05-08T23:32:17.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7feacf7fe700 1 -- 172.21.15.33:0/1332854157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00036a0 con 0x7fead00b3200 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.546+0000 7ff9d1901700 1 -- start start 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0713d0 0x7ff9cc1f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0c3020 0x7ff9cc1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc0c9050 0x7ff9cc200ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc204110 con 0x7ff9cc0c9050 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0bfbf0 con 0x7ff9cc0713d0 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7ff9d1901700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0bfed0 con 0x7ff9cc0c3020 2024-05-08T23:32:17.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7f952c778700 1 -- start start 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c70c0 0x7f9524163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95240c8e50 0x7f95241fc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9524156db0 0x7f9524200630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524151d60 con 0x7f95240c8e50 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95241ffb90 con 0x7f9524156db0 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f952c778700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9524164240 con 0x7f95240c70c0 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fe19b2af700 1 -- 172.21.15.33:0/1727913595 shutdown_connections 2024-05-08T23:32:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fe19b2af700 1 --2- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe194070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fe19b2af700 1 --2- 172.21.15.33:0/1727913595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe1940c21a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fe19b2af700 1 -- 172.21.15.33:0/1727913595 >> 172.21.15.33:0/1727913595 conn(0x7fe1940c8360 msgr2=0x7fe1940c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.547+0000 7fefe7194700 1 -- start start 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefe00c6260 0x7fefe015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01504a0 0x7fefe0155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe0156f50 0x7fefe01f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0154900 con 0x7fefe01504a0 2024-05-08T23:32:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0154bc0 con 0x7fefe00c6260 2024-05-08T23:32:17.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fefe7194700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0154ea0 con 0x7fefe0156f50 2024-05-08T23:32:17.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f748c558700 1 Processor -- start 2024-05-08T23:32:17.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7f748c558700 1 -- start start 2024-05-08T23:32:17.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fead725f700 1 -- 172.21.15.33:0/1332854157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead00b3200 msgr2=0x7fead00cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fead725f700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead00b3200 0x7fead00cd070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feac0009840 tx=0x7feac0009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0713d0 0x7ff9cc1f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9caffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0713d0 0x7ff9cc1f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54836/0 (socket says 172.21.15.33:54836) 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9caffd700 1 -- 172.21.15.33:0/308844739 learned_addr learned my addr 172.21.15.33:0/308844739 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0c3020 0x7ff9cc1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 -- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0713d0 msgr2=0x7ff9cc1f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9cc0713d0 0x7ff9cc1f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 -- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc0c9050 msgr2=0x7ff9cc200ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9cc0c9050 0x7ff9cc200ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7ff9ca7fc700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0009550 con 0x7ff9cc0c3020 2024-05-08T23:32:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7ff9ca7fc700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0c3020 0x7ff9cc1644b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c8c0 tx=0x7ff9c400cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fefe4f30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefe00c6260 0x7fefe015a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fefe5731700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe0156f50 0x7fefe01f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fefe5731700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe0156f50 0x7fefe01f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53568/0 (socket says 172.21.15.33:53568) 2024-05-08T23:32:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fefe5731700 1 -- 172.21.15.33:0/1868713265 learned_addr learned my addr 172.21.15.33:0/1868713265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fa31130c700 1 Processor -- start 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4003e30 con 0x7ff9cc0c3020 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4005070 con 0x7ff9cc0c3020 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc204360 con 0x7ff9cc0c3020 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc204920 con 0x7ff9cc0c3020 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f952ad15700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9524156db0 0x7f9524200630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f952a514700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c70c0 0x7f9524163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f952ad15700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9524156db0 0x7f9524200630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54838/0 (socket says 172.21.15.33:54838) 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f952ad15700 1 -- 172.21.15.33:0/1015140435 learned_addr learned my addr 172.21.15.33:0/1015140435 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f952a514700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9524156db0 msgr2=0x7f9524200630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f952a514700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9524156db0 0x7f9524200630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f952a514700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95240c8e50 msgr2=0x7f95241fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f952a514700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95240c8e50 0x7f95241fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f952a514700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009b30 con 0x7f95240c70c0 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fe19b2af700 1 -- 172.21.15.33:0/1727913595 shutdown_connections 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.548+0000 7fe19b2af700 1 -- 172.21.15.33:0/1727913595 wait complete. 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fe19b2af700 1 Processor -- start 2024-05-08T23:32:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f74840c2770 0x7f74840c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840c7d80 0x7f74841f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 0x7f74841f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841f4e30 con 0x7f74840c7d80 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841f3fd0 con 0x7f74841589d0 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748c558700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74841f42b0 con 0x7f74840c2770 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748aaf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 0x7f74841f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748aaf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 0x7f74841f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54854/0 (socket says 172.21.15.33:54854) 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7f748aaf5700 1 -- 172.21.15.33:0/4146845874 learned_addr learned my addr 172.21.15.33:0/4146845874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748a2f4700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f74840c2770 0x7f74840c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f7489af3700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840c7d80 0x7f74841f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748aaf5700 1 -- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f74840c2770 msgr2=0x7f74840c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748aaf5700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f74840c2770 0x7f74840c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748aaf5700 1 -- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840c7d80 msgr2=0x7f74841f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748aaf5700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840c7d80 0x7f74841f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f748aaf5700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740094a0 con 0x7f74841589d0 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fa31130c700 1 -- start start 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fead725f700 1 -- 172.21.15.33:0/1332854157 shutdown_connections 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fead725f700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead00b3200 0x7fead00cd070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fead725f700 1 --2- 172.21.15.33:0/1332854157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.549+0000 7fead725f700 1 -- 172.21.15.33:0/1332854157 >> 172.21.15.33:0/1332854157 conn(0x7fead006cd80 msgr2=0x7fead006d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4005400 con 0x7ff9cc0c3020 2024-05-08T23:32:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7f7489af3700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74840c7d80 0x7f74841f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fa31130c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa30c150510 0x7fa30c0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fa31130c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa30c0c70c0 0x7fa30c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9ac000fc0 con 0x7ff9cc0c3020 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefdffff700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01504a0 0x7fefe0155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefe4f30700 1 -- 172.21.15.33:0/1868713265 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe0156f50 msgr2=0x7fefe01f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefe4f30700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fefe0156f50 0x7fefe01f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefe4f30700 1 -- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01504a0 msgr2=0x7fefe0155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefe4f30700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefe01504a0 0x7fefe0155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fefe4f30700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009b30 con 0x7fefe00c6260 2024-05-08T23:32:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7f952a514700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c70c0 0x7f9524163c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f951c014000 tx=0x7f951c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa31130c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c06b220 con 0x7fa30c0c8e50 2024-05-08T23:32:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa31130c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0c8110 con 0x7fa30c150510 2024-05-08T23:32:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa31130c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0c7e30 con 0x7fa30c0c70c0 2024-05-08T23:32:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.550+0000 7fe19b2af700 1 -- start start 2024-05-08T23:32:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fefe4f30700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefe00c6260 0x7fefe015a270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefd8015000 tx=0x7fefd8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7f748aaf5700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 0x7f74841f7ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f747c00efc0 tx=0x7f747c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30b7fe700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa30c0c8e50 0x7fa30c1598a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59012/0 (socket says 172.21.15.33:59012) 2024-05-08T23:32:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa30c150510 0x7fa30c0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1104006548 learned_addr learned my addr 172.21.15.33:0/1104006548 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30a7fc700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa30c0c70c0 0x7fa30c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fead725f700 1 -- 172.21.15.33:0/1332854157 shutdown_connections 2024-05-08T23:32:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fead725f700 1 -- 172.21.15.33:0/1332854157 wait complete. 2024-05-08T23:32:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8004250 con 0x7fefe00c6260 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30a7fc700 1 -- 172.21.15.33:0/1104006548 >> v2:172.21.15.33:3300/0 conn(0x7fa30c150510 msgr2=0x7fa30c0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30a7fc700 1 --2- 172.21.15.33:0/1104006548 >> v2:172.21.15.33:3300/0 conn(0x7fa30c150510 0x7fa30c0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30a7fc700 1 -- 172.21.15.33:0/1104006548 >> v1:172.21.15.42:6789/0 conn(0x7fa30c0c8e50 legacy=0x7fa30c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa309ffb700 1 -- 172.21.15.33:0/1104006548 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870297968 0 0) 0x7fa30c06b220 con 0x7fa30c0c8e50 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fa30a7fc700 1 -- 172.21.15.33:0/1104006548 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0c6300 con 0x7fa30c0c70c0 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80043b0 con 0x7fefe00c6260 2024-05-08T23:32:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.551+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8004760 con 0x7fefe00c6260 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1940c2770 0x7fe194155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe19414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe194150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194155450 con 0x7fe1940c2770 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194154e90 con 0x7fe1941589d0 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19b2af700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe194155170 con 0x7fe1940c7d80 2024-05-08T23:32:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fa30a7fc700 1 --2- 172.21.15.33:0/1104006548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c70c0 0x7fa30c1577a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa30c06bbe0 tx=0x7fa300009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7c4629ffd79936 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c009ea0 con 0x7f74841589d0 2024-05-08T23:32:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00f040 con 0x7f74841589d0 2024-05-08T23:32:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00ced0 con 0x7f74841589d0 2024-05-08T23:32:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74841f4590 con 0x7f74841589d0 2024-05-08T23:32:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe19414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe19414fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53590/0 (socket says 172.21.15.33:53590) 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 -- 172.21.15.33:0/2788456830 learned_addr learned my addr 172.21.15.33:0/2788456830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19984c700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe194150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 -- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 msgr2=0x7fe194150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1941589d0 0x7fe194150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 -- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1940c2770 msgr2=0x7fe194155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1940c2770 0x7fe194155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7fe19884a700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900094a0 con 0x7fe1940c7d80 2024-05-08T23:32:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe0155180 con 0x7fefe00c6260 2024-05-08T23:32:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00c5de0 con 0x7fefe00c6260 2024-05-08T23:32:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fe19884a700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe19414fd90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c4d0 tx=0x7fe18400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184011790 con 0x7fe1940c7d80 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fa309ffb700 1 -- 172.21.15.33:0/1104006548 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300010a90 con 0x7fa30c0c70c0 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fa309ffb700 1 -- 172.21.15.33:0/1104006548 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300010bf0 con 0x7fa30c0c70c0 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fa309ffb700 1 -- 172.21.15.33:0/1104006548 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300018440 con 0x7fa30c0c70c0 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1941f4990 con 0x7fe1940c7d80 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1941f4f50 con 0x7fe1940c7d80 2024-05-08T23:32:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7fead725f700 1 Processor -- start 2024-05-08T23:32:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c004440 con 0x7f95240c70c0 2024-05-08T23:32:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fefe0157390 con 0x7fefe00c6260 2024-05-08T23:32:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.553+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9c4005560 con 0x7ff9cc0c3020 2024-05-08T23:32:17.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7ff9b3fff700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff9b806ad60 0x7ff9b806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9c40891b0 con 0x7ff9cc0c3020 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 -- 172.21.15.33:0/1104006548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c70c0 msgr2=0x7fa30c1577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 --2- 172.21.15.33:0/1104006548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c70c0 0x7fa30c1577a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa30c06bbe0 tx=0x7fa300009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7ff9b3fff700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7ff9b8072f80 0x7ff9b80753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9b8075a50 con 0x7ff9b8072f80 2024-05-08T23:32:17.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c4089560 con 0x7ff9cc0c3020 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.552+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841f4b50 con 0x7f74841589d0 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 -- 172.21.15.33:0/1104006548 shutdown_connections 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 --2- 172.21.15.33:0/1104006548 >> v2:172.21.15.33:3300/0 conn(0x7fa30c150510 0x7fa30c0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 --2- 172.21.15.33:0/1104006548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c70c0 0x7fa30c1577a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fa31130c700 1 -- 172.21.15.33:0/1104006548 >> 172.21.15.33:0/1104006548 conn(0x7fa30c0c8360 msgr2=0x7fa30c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f748406b760 con 0x7f74841589d0 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c0045e0 con 0x7f95240c70c0 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c012430 con 0x7f95240c70c0 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9524164520 con 0x7f95240c70c0 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95241ffe30 con 0x7f95240c70c0 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840118f0 con 0x7fe1940c7d80 2024-05-08T23:32:17.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.554+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe174000fc0 con 0x7fe1940c7d80 2024-05-08T23:32:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.555+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fefd801b430 con 0x7fefe00c6260 2024-05-08T23:32:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.555+0000 7ff9cb7fe700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7ff9b8072f80 0x7ff9b80753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.555+0000 7ff9cb7fe700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7ff9b8072f80 0x7ff9b80753a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.555+0000 7ff9caffd700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff9b806ad60 0x7ff9b806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184011ca0 con 0x7fe1940c7d80 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f950c000fc0 con 0x7f95240c70c0 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdd7fa700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fefcc06ad10 0x7fefcc06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefd808a870 con 0x7fefe00c6260 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdffff700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fefcc06ad10 0x7fefcc06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdd7fa700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fefcc072f30 0x7fefcc075350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefcc075a00 con 0x7fefcc072f30 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd808abe0 con 0x7fefe00c6260 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f747c014480 con 0x7f74841589d0 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7f747affd700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f746806acc0 0x7f746806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f747c089b60 con 0x7f74841589d0 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7f747affd700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f7468072cf0 0x7f7468075110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74680757c0 con 0x7f7468072cf0 2024-05-08T23:32:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7fead725f700 1 -- start start 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.555+0000 7fa31130c700 1 -- 172.21.15.33:0/1104006548 shutdown_connections 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.556+0000 7fa31130c700 1 -- 172.21.15.33:0/1104006548 wait complete. 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7fa31130c700 1 Processor -- start 2024-05-08T23:32:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.557+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== osd.7 v2:172.21.15.42:6808/47101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff9b8075a50 con 0x7ff9b8072f80 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead00b3200 0x7fead0200650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 0x7fead01fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead015e340 con 0x7fead01fb160 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead01fa710 con 0x7fead00b3200 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead725f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead01fbb30 con 0x7fead015f460 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead57fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 0x7fead01fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead57fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 0x7fead01fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52890/0 (socket says 172.21.15.33:52890) 2024-05-08T23:32:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fead57fc700 1 -- 172.21.15.33:0/792088632 learned_addr learned my addr 172.21.15.33:0/792088632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f951c019070 con 0x7f95240c70c0 2024-05-08T23:32:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7f951affd700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f950806ad90 0x7f950806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f951c08a3a0 con 0x7f95240c70c0 2024-05-08T23:32:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead4ffb700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead00b3200 0x7fead0200650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7f951affd700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9508075a00 con 0x7f9508072f30 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7f748a2f4700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f746806acc0 0x7f746806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fa31130c700 1 -- start start 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fa31130c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa30c0c70c0 0x7fa30c14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.558+0000 7fa31130c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa30c0c8e50 0x7fa30c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa31130c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 0x7fa30c071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa31130c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1541c0 con 0x7fa30c0c70c0 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa31130c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c155050 con 0x7fa30c0c1c80 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa31130c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1513e0 con 0x7fa30c0c8e50 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 0x7fa30c071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 0x7fa30c071ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54884/0 (socket says 172.21.15.33:54884) 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1350637959 learned_addr learned my addr 172.21.15.33:0/1350637959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30a7fc700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa30c0c8e50 0x7fa30c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7feacffff700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead57fc700 1 -- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 msgr2=0x7fead00c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead57fc700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fead015f460 0x7fead00c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead57fc700 1 -- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead00b3200 msgr2=0x7fead0200650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead57fc700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fead00b3200 0x7fead0200650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fead57fc700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00094a0 con 0x7fead01fb160 2024-05-08T23:32:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7f9529d13700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f950806ad90 0x7f950806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7f7489af3700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f7468072cf0 0x7f7468075110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa30c0c8e50 msgr2=0x7fa30c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa30c0c8e50 0x7fa30c0c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa30c0c70c0 msgr2=0x7fa30c14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa30c0c70c0 0x7fa30c14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.559+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094a0 con 0x7fa30c0c1c80 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa30b7fe700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 0x7fa30c071ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa30400cca0 tx=0x7fa30400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30400ab20 con 0x7fa30c0c1c80 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30400aca0 con 0x7fa30c0c1c80 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c14ba70 con 0x7fa30c0c1c80 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c151100 con 0x7fa30c0c1c80 2024-05-08T23:32:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304007310 con 0x7fa30c0c1c80 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fead57fc700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 0x7fead01fb5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fead008d8f0 tx=0x7feac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00045e0 con 0x7fead01fb160 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac0011630 con 0x7fead01fb160 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00119c0 con 0x7fead01fb160 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7ff9caffd700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff9b806ad60 0x7ff9b806d220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0004720 tx=0x7ff9c0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7fefe5731700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fefcc072f30 0x7fefcc075350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead0200b90 con 0x7fead01fb160 2024-05-08T23:32:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead015e570 con 0x7fead01fb160 2024-05-08T23:32:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fead006aee0 con 0x7fead01fb160 2024-05-08T23:32:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7f7489af3700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f7468072cf0 0x7f7468075110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.562+0000 7fefdffff700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fefcc06ad10 0x7fefcc06d1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefe0146160 tx=0x7fefd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.562+0000 7f748a2f4700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f746806acc0 0x7f746806d180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7480005910 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.562+0000 7fefe5731700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fefcc072f30 0x7fefcc075350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== osd.3 v2:172.21.15.33:6824/47140 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f74680757c0 con 0x7f7468072cf0 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7f9529d13700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f950806ad90 0x7f950806d250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9524152340 tx=0x7f9520008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f951c0555f0 con 0x7f95240c70c0 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7f952ad15700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7f952ad15700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 msgr2=0x7f9508075350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7f952ad15700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 msgr2=0x7f9508075350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7f952ad15700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 crc :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7f952ad15700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe18401c460 con 0x7fe1940c7d80 2024-05-08T23:32:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fe189ffb700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe17806acc0 0x7fe17806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.560+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe184089be0 con 0x7fe1940c7d80 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7fe189ffb700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fe178072e60 0x7fe178075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe178075930 con 0x7fe178072e60 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.561+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe184089f90 con 0x7fe1940c7d80 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7fe19984c700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fe178072e60 0x7fe178075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7fe19904b700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe17806acc0 0x7fe17806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7fe19984c700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fe178072e60 0x7fe178075280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7fe19904b700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe17806acc0 0x7fe17806d180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1900090b0 tx=0x7fe190009f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== osd.2 v2:172.21.15.33:6808/47118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fefcc075a00 con 0x7fefcc072f30 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747c054d30 con 0x7f74841589d0 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa304007470 con 0x7fa30c0c1c80 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2ec000fc0 con 0x7fa30c0c1c80 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7fa2f37fe700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa2f406aed0 0x7fa2f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa304089180 con 0x7fa30c0c1c80 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7fa2f37fe700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7fa2f4073070 0x7fa2f4075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2f4075b40 con 0x7fa2f4073070 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.563+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feac0018040 con 0x7fead01fb160 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7feacd7fa700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feabc06ad60 0x7feabc06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feac008a790 con 0x7fead01fb160 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7feacd7fa700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7feabc072e10 0x7feabc075230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feabc0758e0 con 0x7feabc072e10 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feac008ab40 con 0x7fead01fb160 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.564+0000 7fead4ffb700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feabc06ad60 0x7feabc06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7feacffff700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7feabc072e10 0x7feabc075230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7fa30a7fc700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7fa2f4073070 0x7fa2f4075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7fa30affd700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa2f406aed0 0x7fa2f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7feacffff700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7feabc072e10 0x7feabc075230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.565+0000 7fead4ffb700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feabc06ad60 0x7feabc06d220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feac8009fd0 tx=0x7feac800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3040543d0 con 0x7fa30c0c1c80 2024-05-08T23:32:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== osd.1 v2:172.21.15.33:6816/47120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe178075930 con 0x7fe178072e60 2024-05-08T23:32:17.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7fa30a7fc700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7fa2f4073070 0x7fa2f4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7fa30affd700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa2f406aed0 0x7fa2f406d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006b20 tx=0x7fa2fc006ab0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7f66d80a0700 1 Processor -- start 2024-05-08T23:32:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== osd.6 v2:172.21.15.42:6816/47123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7feabc0758e0 con 0x7feabc072e10 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.566+0000 7f66d80a0700 1 -- start start 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.567+0000 7f66d80a0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d00c70c0 0x7f66d00c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.567+0000 7f66d80a0700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d0157c30 con 0x7f66d0150510 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.567+0000 7f66d80a0700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d00c8110 con 0x7f66d01577a0 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.567+0000 7f66d80a0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00c7e30 con 0x7f66d00c70c0 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.568+0000 7f66d5e3c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f66d0150510 0x7f66d006a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51976/0 (socket says 172.21.15.33:51976) 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.568+0000 7f66d5e3c700 1 -- 172.21.15.33:0/1693370626 learned_addr learned my addr 172.21.15.33:0/1693370626 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.568+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892155436 0 0) 0x7f66d0157c30 con 0x7f66d0150510 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.568+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66bc0035b0 con 0x7f66d0150510 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.568+0000 7f66d563b700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d00c70c0 0x7f66d00c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067595826 0 0) 0x7f66bc0035b0 con 0x7f66d0150510 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d00c70c0 msgr2=0x7f66d00c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d00c70c0 0x7f66d00c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 >> v1:172.21.15.42:6789/0 conn(0x7f66d01577a0 legacy=0x7f66d0160bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00c6300 con 0x7f66d0150510 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f66cc001670 con 0x7f66d0150510 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 >> v1:172.21.15.33:6789/0 conn(0x7f66d0150510 legacy=0x7f66d006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66bc003a30 0x7f66bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66bc004850 0x7f66bc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc005210 0x7f66bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00c7e30 con 0x7f66bc005210 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc00c110 con 0x7f66bc004850 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc00c2d0 con 0x7f66bc003a30 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66bc003a30 0x7f66bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d663d700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc005210 0x7f66bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 -- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66bc004850 msgr2=0x7f66bc004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66bc004850 0x7f66bc004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 -- 172.21.15.33:0/1693370626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc005210 msgr2=0x7f66bc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66bc005210 0x7f66bc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7f66d5e3c700 1 -- 172.21.15.33:0/1693370626 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0035b0 con 0x7f66bc003a30 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.570+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66bc003a30 0x7f66bc003e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66cc007cb0 tx=0x7f66bc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e624997ec123d845 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.570+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc00cd50 con 0x7f66bc003a30 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.570+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc005960 con 0x7f66bc003a30 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.570+0000 7f66d4e3a700 1 -- 172.21.15.33:0/1693370626 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc005d10 con 0x7f66bc003a30 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.571+0000 7f66d80a0700 1 -- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66bc003a30 msgr2=0x7f66bc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.571+0000 7f66d80a0700 1 --2- 172.21.15.33:0/1693370626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66bc003a30 0x7f66bc003e30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f66cc007cb0 tx=0x7f66bc005880 comp rx=0 tx=0).stop 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.571+0000 7f66d663d700 1 -- 172.21.15.33:0/1693370626 reap_dead start 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.571+0000 7f66d80a0700 1 -- 172.21.15.33:0/1693370626 shutdown_connections 2024-05-08T23:32:17.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.571+0000 7f66d80a0700 1 -- 172.21.15.33:0/1693370626 >> 172.21.15.33:0/1693370626 conn(0x7f66d00c8360 msgr2=0x7f66d00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.569+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== osd.4 v2:172.21.15.42:6823/47110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa2f4075b40 con 0x7fa2f4073070 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.577+0000 7f66d80a0700 1 -- 172.21.15.33:0/1693370626 shutdown_connections 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.577+0000 7f66d80a0700 1 -- 172.21.15.33:0/1693370626 wait complete. 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.579+0000 7f66d80a0700 1 Processor -- start 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.579+0000 7f66d80a0700 1 -- start start 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66d014ce50 0x7f66d014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d014d230 0x7f66d0148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d0200220 0x7f66d0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01494b0 con 0x7f66d0200220 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0151770 con 0x7f66d014ce50 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.580+0000 7f66d80a0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0150c80 con 0x7f66d014d230 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d563b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d014d230 0x7f66d0148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66d014ce50 0x7f66d014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d563b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d014d230 0x7f66d0148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53634/0 (socket says 172.21.15.33:53634) 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d563b700 1 -- 172.21.15.33:0/306395594 learned_addr learned my addr 172.21.15.33:0/306395594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d663d700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d0200220 0x7f66d0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 -- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d014d230 msgr2=0x7f66d0148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f66d014d230 0x7f66d0148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 -- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d0200220 msgr2=0x7f66d0152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d0200220 0x7f66d0152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.581+0000 7f66d5e3c700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0148ae0 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66d5e3c700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66d014ce50 0x7f66d014a6f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66cc015000 tx=0x7f66cc001930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc005cd0 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc005e30 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0072f0 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d01527d0 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.582+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0150ed0 con 0x7f66d014ce50 2024-05-08T23:32:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.583+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66d006cf00 con 0x7f66d014ce50 2024-05-08T23:32:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.586+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f66cc020020 con 0x7f66d014ce50 2024-05-08T23:32:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.586+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 --> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe174002d40 con 0x7fe178072e60 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.586+0000 7f66c67fc700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f66b406acc0 0x7f66b406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.586+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66cc08a0d0 con 0x7f66d014ce50 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.586+0000 7f66c67fc700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7f66b4072e60 0x7f66b4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.587+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66b4075930 con 0x7f66b4072e60 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.587+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66cc08a440 con 0x7f66d014ce50 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.587+0000 7f66d563b700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f66b406acc0 0x7f66b406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.587+0000 7f66d663d700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7f66b4072e60 0x7f66b4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.587+0000 7f66d663d700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7f66b4072e60 0x7f66b4075280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.588+0000 7f66d563b700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f66b406acc0 0x7f66b406d180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66d0158cd0 tx=0x7f66c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.588+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== osd.5 v2:172.21.15.42:6800/47100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f66b4075930 con 0x7f66b4072e60 2024-05-08T23:32:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7fe189ffb700 1 -- 172.21.15.33:0/2788456830 <== osd.1 v2:172.21.15.33:6816/47120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe174002d40 con 0x7fe178072e60 2024-05-08T23:32:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.590+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 --> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9ac002d40 con 0x7ff9b8072f80 2024-05-08T23:32:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.591+0000 7ff9b3fff700 1 -- 172.21.15.33:0/308844739 <== osd.7 v2:172.21.15.42:6808/47101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9ac002d40 con 0x7ff9b8072f80 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.591+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7ff9b8072f80 msgr2=0x7ff9b80753a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7ff9b8072f80 0x7ff9b80753a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff9b806ad60 msgr2=0x7ff9b806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff9b806ad60 0x7ff9b806d220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0004720 tx=0x7ff9c0008030 comp rx=0 tx=0).stop 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0c3020 msgr2=0x7ff9cc1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 --2- 172.21.15.33:0/308844739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9cc0c3020 0x7ff9cc1644b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c8c0 tx=0x7ff9c400cbd0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9cb7fe700 1 -- 172.21.15.33:0/308844739 reap_dead start 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 shutdown_connections 2024-05-08T23:32:17.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.592+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 >> 172.21.15.33:0/308844739 conn(0x7ff9cc0c8360 msgr2=0x7ff9cc164d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.593+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 shutdown_connections 2024-05-08T23:32:17.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.593+0000 7ff9d1901700 1 -- 172.21.15.33:0/308844739 wait complete. 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.593+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fe178072e60 msgr2=0x7fe178075280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.593+0000 7fe19b2af700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fe178072e60 0x7fe178075280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.593+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe17806acc0 msgr2=0x7fe17806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe17806acc0 0x7fe17806d180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1900090b0 tx=0x7fe190009f90 comp rx=0 tx=0).stop 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 msgr2=0x7fe19414fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 --2- 172.21.15.33:0/2788456830 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1940c7d80 0x7fe19414fd90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c4d0 tx=0x7fe18400c890 comp rx=0 tx=0).stop 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19984c700 1 -- 172.21.15.33:0/2788456830 reap_dead start 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 shutdown_connections 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 >> 172.21.15.33:0/2788456830 conn(0x7fe1940c8360 msgr2=0x7fe19415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 shutdown_connections 2024-05-08T23:32:17.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.594+0000 7fe19b2af700 1 -- 172.21.15.33:0/2788456830 wait complete. 2024-05-08T23:32:17.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.598+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 --> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f748406b4c0 con 0x7f7468072cf0 2024-05-08T23:32:17.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f747affd700 1 -- 172.21.15.33:0/4146845874 <== osd.3 v2:172.21.15.33:6824/47140 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7474022040 con 0x7f7468072cf0 2024-05-08T23:32:17.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 --> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fead00cc1e0 con 0x7feabc072e10 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f7468072cf0 msgr2=0x7f7468075110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f7468072cf0 0x7f7468075110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f746806acc0 msgr2=0x7f746806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f746806acc0 0x7f746806d180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7480005910 tx=0x7f7480009040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 msgr2=0x7f74841f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748c558700 1 --2- 172.21.15.33:0/4146845874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74841589d0 0x7f74841f7ca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f747c00efc0 tx=0x7f747c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.600+0000 7f748aaf5700 1 -- 172.21.15.33:0/4146845874 reap_dead start 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 shutdown_connections 2024-05-08T23:32:17.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 >> 172.21.15.33:0/4146845874 conn(0x7f74840c8360 msgr2=0x7f74840c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7feacd7fa700 1 -- 172.21.15.33:0/792088632 <== osd.6 v2:172.21.15.42:6816/47123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feac4004870 con 0x7feabc072e10 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7feabc072e10 msgr2=0x7feabc075230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7feabc072e10 0x7feabc075230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feabc06ad60 msgr2=0x7feabc06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feabc06ad60 0x7feabc06d220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feac8009fd0 tx=0x7feac800b040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 msgr2=0x7fead01fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.601+0000 7fead725f700 1 --2- 172.21.15.33:0/792088632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead01fb160 0x7fead01fb5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fead008d8f0 tx=0x7feac0004750 comp rx=0 tx=0).stop 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.602+0000 7fead57fc700 1 -- 172.21.15.33:0/792088632 reap_dead start 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.602+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 shutdown_connections 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.602+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 >> 172.21.15.33:0/792088632 conn(0x7fead006cd80 msgr2=0x7fead006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.602+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 shutdown_connections 2024-05-08T23:32:17.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.602+0000 7fead725f700 1 -- 172.21.15.33:0/792088632 wait complete. 2024-05-08T23:32:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.604+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 shutdown_connections 2024-05-08T23:32:17.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.607+0000 7f748c558700 1 -- 172.21.15.33:0/4146845874 wait complete. 2024-05-08T23:32:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.607+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 --> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fefe00713b0 con 0x7fefcc072f30 2024-05-08T23:32:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.608+0000 7fefdd7fa700 1 -- 172.21.15.33:0/1868713265 <== osd.2 v2:172.21.15.33:6808/47118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fefd0009ad0 con 0x7fefcc072f30 2024-05-08T23:32:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 --> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa2ec002d40 con 0x7fa2f4073070 2024-05-08T23:32:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fefcc072f30 msgr2=0x7fefcc075350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fefcc072f30 0x7fefcc075350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fefcc06ad10 msgr2=0x7fefcc06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fefcc06ad10 0x7fefcc06d1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefe0146160 tx=0x7fefd4008040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefe00c6260 msgr2=0x7fefe015a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe7194700 1 --2- 172.21.15.33:0/1868713265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefe00c6260 0x7fefe015a270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefd8015000 tx=0x7fefd8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fefe5731700 1 -- 172.21.15.33:0/1868713265 reap_dead start 2024-05-08T23:32:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.610+0000 7fa2f37fe700 1 -- 172.21.15.33:0/1350637959 <== osd.4 v2:172.21.15.42:6823/47110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa2ec002d40 con 0x7fa2f4073070 2024-05-08T23:32:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 shutdown_connections 2024-05-08T23:32:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 >> 172.21.15.33:0/1868713265 conn(0x7fefe00c8360 msgr2=0x7fefe00c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7fa2f4073070 msgr2=0x7fa2f4075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fa31130c700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7fa2f4073070 0x7fa2f4075490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 shutdown_connections 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.611+0000 7fefe7194700 1 -- 172.21.15.33:0/1868713265 wait complete. 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.612+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa2f406aed0 msgr2=0x7fa2f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.612+0000 7fa31130c700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa2f406aed0 0x7fa2f406d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006b20 tx=0x7fa2fc006ab0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.612+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 msgr2=0x7fa30c071ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.612+0000 7fa31130c700 1 --2- 172.21.15.33:0/1350637959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0c1c80 0x7fa30c071ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa30400cca0 tx=0x7fa30400a060 comp rx=0 tx=0).stop 2024-05-08T23:32:17.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.612+0000 7fa30b7fe700 1 -- 172.21.15.33:0/1350637959 reap_dead start 2024-05-08T23:32:17.633 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:32:17.634 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:32:17.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.617+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 shutdown_connections 2024-05-08T23:32:17.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.617+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 >> 172.21.15.33:0/1350637959 conn(0x7fa30c0c8360 msgr2=0x7fa30c0c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.617+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 shutdown_connections 2024-05-08T23:32:17.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.617+0000 7fa31130c700 1 -- 172.21.15.33:0/1350637959 wait complete. 2024-05-08T23:32:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.619+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 --> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66d00c6ba0 con 0x7f66b4072e60 2024-05-08T23:32:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.621+0000 7f66c67fc700 1 -- 172.21.15.33:0/306395594 <== osd.5 v2:172.21.15.42:6800/47100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f66c8009af0 con 0x7f66b4072e60 2024-05-08T23:32:17.636 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-05-08T23:32:17.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7f66b4072e60 msgr2=0x7f66b4075280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7f66b4072e60 0x7f66b4075280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f66b406acc0 msgr2=0x7f66b406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f66b406acc0 0x7f66b406d180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66d0158cd0 tx=0x7f66c0008040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66d014ce50 msgr2=0x7f66d014a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d80a0700 1 --2- 172.21.15.33:0/306395594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66d014ce50 0x7f66d014a6f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f66cc015000 tx=0x7f66cc001930 comp rx=0 tx=0).stop 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.623+0000 7f66d663d700 1 -- 172.21.15.33:0/306395594 reap_dead start 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.624+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 shutdown_connections 2024-05-08T23:32:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.624+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 >> 172.21.15.33:0/306395594 conn(0x7f66d00c8360 msgr2=0x7f66d00c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.624+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 shutdown_connections 2024-05-08T23:32:17.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.624+0000 7f66d80a0700 1 -- 172.21.15.33:0/306395594 wait complete. 2024-05-08T23:32:17.639 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:32:17.640 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:32:17.642 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-05-08T23:32:17.642 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:32:17.643 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:32:17.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.765+0000 7f952ad15700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.766+0000 7f952ad15700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.767+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== osd.0 v2:172.21.15.33:6800/47117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9508075a00 con 0x7f9508072f30 2024-05-08T23:32:17.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.799+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 --> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f950c002d40 con 0x7f9508072f30 2024-05-08T23:32:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.800+0000 7f951affd700 1 -- 172.21.15.33:0/1015140435 <== osd.0 v2:172.21.15.33:6800/47117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f950c002d40 con 0x7f9508072f30 2024-05-08T23:32:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 msgr2=0x7f9508075350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f9508072f30 0x7f9508075350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f950806ad90 msgr2=0x7f950806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f950806ad90 0x7f950806d250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9524152340 tx=0x7f9520008040 comp rx=0 tx=0).stop 2024-05-08T23:32:17.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c70c0 msgr2=0x7f9524163c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952c778700 1 --2- 172.21.15.33:0/1015140435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f95240c70c0 0x7f9524163c20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f951c014000 tx=0x7f951c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.801+0000 7f952ad15700 1 -- 172.21.15.33:0/1015140435 reap_dead start 2024-05-08T23:32:17.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.803+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 shutdown_connections 2024-05-08T23:32:17.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.803+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 >> 172.21.15.33:0/1015140435 conn(0x7f95240c8360 msgr2=0x7f95240c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.803+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 shutdown_connections 2024-05-08T23:32:17.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.803+0000 7f952c778700 1 -- 172.21.15.33:0/1015140435 wait complete. 2024-05-08T23:32:17.828 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:17.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:32:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.838+0000 7f2b044d3700 1 Processor -- start 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.838+0000 7f2b044d3700 1 -- start start 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.840+0000 7f2b044d3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2afc0c70d0 0x7f2afc0c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.840+0000 7f2b044d3700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2afc161c70 0x7f2afc1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.840+0000 7f2b044d3700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc158ee0 con 0x7f2afc0722c0 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.840+0000 7f2b044d3700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0c2e60 con 0x7f2afc0c70d0 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.840+0000 7f2b044d3700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc06c730 con 0x7f2afc161c70 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2afc0c70d0 0x7f2afc0c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b0226f700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2afc0722c0 0x7f2afc0c69c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51982/0 (socket says 172.21.15.33:51982) 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2afc0c70d0 0x7f2afc0c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52914/0 (socket says 172.21.15.33:52914) 2024-05-08T23:32:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b0226f700 1 -- 172.21.15.33:0/3927032541 learned_addr learned my addr 172.21.15.33:0/3927032541 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 -- 172.21.15.33:0/3927032541 >> v2:172.21.15.42:3300/0 conn(0x7f2afc161c70 msgr2=0x7f2afc1620a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 --2- 172.21.15.33:0/3927032541 >> v2:172.21.15.42:3300/0 conn(0x7f2afc161c70 0x7f2afc1620a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 -- 172.21.15.33:0/3927032541 >> v1:172.21.15.33:6789/0 conn(0x7f2afc0722c0 legacy=0x7f2afc0c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.841+0000 7f2b01a6e700 1 -- 172.21.15.33:0/3927032541 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc06ca10 con 0x7f2afc0c70d0 2024-05-08T23:32:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.842+0000 7f2b01a6e700 1 --2- 172.21.15.33:0/3927032541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc0c70d0 0x7f2afc0c94e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06fd40 tx=0x7f2af400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743ed0933ee5849f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.842+0000 7f2b0126d700 1 -- 172.21.15.33:0/3927032541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400e070 con 0x7f2afc0c70d0 2024-05-08T23:32:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.842+0000 7f2b0126d700 1 -- 172.21.15.33:0/3927032541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af400bad0 con 0x7f2afc0c70d0 2024-05-08T23:32:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.842+0000 7f2b0126d700 1 -- 172.21.15.33:0/3927032541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400be80 con 0x7f2afc0c70d0 2024-05-08T23:32:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 -- 172.21.15.33:0/3927032541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc0c70d0 msgr2=0x7f2afc0c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3927032541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc0c70d0 0x7f2afc0c94e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06fd40 tx=0x7f2af400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:17.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 -- 172.21.15.33:0/3927032541 shutdown_connections 2024-05-08T23:32:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3927032541 >> v2:172.21.15.42:3300/0 conn(0x7f2afc161c70 0x7f2afc1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3927032541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc0c70d0 0x7f2afc0c94e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 -- 172.21.15.33:0/3927032541 >> 172.21.15.33:0/3927032541 conn(0x7f2afc0c89c0 msgr2=0x7f2afc0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.844+0000 7f2b044d3700 1 -- 172.21.15.33:0/3927032541 shutdown_connections 2024-05-08T23:32:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.845+0000 7f2b044d3700 1 -- 172.21.15.33:0/3927032541 wait complete. 2024-05-08T23:32:17.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.847+0000 7f2b044d3700 1 Processor -- start 2024-05-08T23:32:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.847+0000 7f2b044d3700 1 -- start start 2024-05-08T23:32:17.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 0x7f2afc146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2afc0c70d0 0x7f2afc141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 0x7f2afc142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0b16f0 con 0x7f2afc161c70 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc143370 con 0x7f2afc0722c0 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.848+0000 7f2b044d3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc147cf0 con 0x7f2afc0c70d0 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.849+0000 7f2b0226f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 0x7f2afc146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.849+0000 7f2b0226f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 0x7f2afc146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54908/0 (socket says 172.21.15.33:54908) 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.849+0000 7f2b0226f700 1 -- 172.21.15.33:0/3987086458 learned_addr learned my addr 172.21.15.33:0/3987086458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.849+0000 7f2b02a70700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 0x7f2afc142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 -- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2afc0c70d0 msgr2=0x7f2afc141030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2afc0c70d0 0x7f2afc141030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 -- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 msgr2=0x7f2afc146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 0x7f2afc146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 -- 172.21.15.33:0/3987086458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4009a70 con 0x7f2afc161c70 2024-05-08T23:32:17.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.850+0000 7f2b02a70700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 0x7f2afc142e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b3c0 tx=0x7f2aec00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.851+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011830 con 0x7f2afc161c70 2024-05-08T23:32:17.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.851+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec011990 con 0x7f2afc161c70 2024-05-08T23:32:17.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.851+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec011d40 con 0x7f2afc161c70 2024-05-08T23:32:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.852+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc0b1940 con 0x7f2afc161c70 2024-05-08T23:32:17.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.852+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc140830 con 0x7f2afc161c70 2024-05-08T23:32:17.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.854+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc075460 con 0x7f2afc161c70 2024-05-08T23:32:17.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.855+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2aec01a400 con 0x7f2afc161c70 2024-05-08T23:32:17.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.856+0000 7f2af2ffd700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 0x7f2ae006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.856+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aec089190 con 0x7f2afc161c70 2024-05-08T23:32:17.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.857+0000 7f2b0226f700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 0x7f2ae006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.857+0000 7f2b0226f700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 0x7f2ae006d1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2af80013a0 tx=0x7f2af8001330 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.865+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2aec0543e0 con 0x7f2afc161c70 2024-05-08T23:32:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.871+0000 7f088a25b700 1 Processor -- start 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.871+0000 7f088a25b700 1 -- start start 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f088a25b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f088a25b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cb2f0 con 0x7f088406c240 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f088a25b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cb6c0 con 0x7f08840b3200 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f088a25b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08840cb940 con 0x7f088415f460 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f08837fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f08837fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53650/0 (socket says 172.21.15.33:53650) 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.872+0000 7f08837fe700 1 -- 172.21.15.33:0/3836167626 learned_addr learned my addr 172.21.15.33:0/3836167626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08837fe700 1 -- 172.21.15.33:0/3836167626 >> v1:172.21.15.42:6789/0 conn(0x7f08840b3200 legacy=0x7f08841691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08837fe700 1 -- 172.21.15.33:0/3836167626 >> v1:172.21.15.33:6789/0 conn(0x7f088406c240 legacy=0x7f088406b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08837fe700 1 -- 172.21.15.33:0/3836167626 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840cbc20 con 0x7f088415f460 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08837fe700 1 --2- 172.21.15.33:0/3836167626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0878009920 tx=0x7f0878009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bb2455a3b0f786 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08827fc700 1 -- 172.21.15.33:0/3836167626 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878004230 con 0x7f088415f460 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.873+0000 7f08827fc700 1 -- 172.21.15.33:0/3836167626 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878004390 con 0x7f088415f460 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f08827fc700 1 -- 172.21.15.33:0/3836167626 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878009e70 con 0x7f088415f460 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f088a25b700 1 -- 172.21.15.33:0/3836167626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 msgr2=0x7f08840ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f088a25b700 1 --2- 172.21.15.33:0/3836167626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0878009920 tx=0x7f0878009c30 comp rx=0 tx=0).stop 2024-05-08T23:32:17.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f088a25b700 1 -- 172.21.15.33:0/3836167626 shutdown_connections 2024-05-08T23:32:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f088a25b700 1 --2- 172.21.15.33:0/3836167626 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088415f460 0x7f08840ccb30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.874+0000 7f088a25b700 1 -- 172.21.15.33:0/3836167626 >> 172.21.15.33:0/3836167626 conn(0x7f088406cd80 msgr2=0x7f088406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.876+0000 7f088a25b700 1 -- 172.21.15.33:0/3836167626 shutdown_connections 2024-05-08T23:32:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.876+0000 7f088a25b700 1 -- 172.21.15.33:0/3836167626 wait complete. 2024-05-08T23:32:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f088a25b700 1 Processor -- start 2024-05-08T23:32:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.877+0000 7f8227247700 1 Processor -- start 2024-05-08T23:32:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.876+0000 7f77d6905700 1 Processor -- start 2024-05-08T23:32:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.876+0000 7f77d6905700 1 -- start start 2024-05-08T23:32:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.877+0000 7f77d6905700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f77d0150510 0x7f77d00c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d6905700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d6905700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d01572f0 con 0x7f77d00c70c0 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d6905700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d0157c30 con 0x7f77d0150510 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d6905700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c8110 con 0x7f77d01577a0 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d4ea2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d4ea2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53654/0 (socket says 172.21.15.33:53654) 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.877+0000 7f8227247700 1 -- start start 2024-05-08T23:32:17.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f8227247700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 0x7f82200b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77d4ea2700 1 -- 172.21.15.33:0/3963380634 learned_addr learned my addr 172.21.15.33:0/3963380634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f8227247700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82200b4340 con 0x7f82200b39e0 2024-05-08T23:32:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f8227247700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822015ed20 con 0x7f822015c000 2024-05-08T23:32:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f8227247700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822015f000 con 0x7f822015e7c0 2024-05-08T23:32:17.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.878+0000 7f77cffff700 1 --2- 172.21.15.33:0/3963380634 >> v2:172.21.15.42:3300/0 conn(0x7f77d0150510 0x7f77d00c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f8224fe3700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f822015c000 0x7f82200b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59030/0 (socket says 172.21.15.33:59030) 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f8224fe3700 1 -- 172.21.15.33:0/356692155 learned_addr learned my addr 172.21.15.33:0/356692155 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77d4ea2700 1 -- 172.21.15.33:0/3963380634 >> v2:172.21.15.42:3300/0 conn(0x7f77d0150510 msgr2=0x7f77d00c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77d4ea2700 1 --2- 172.21.15.33:0/3963380634 >> v2:172.21.15.42:3300/0 conn(0x7f77d0150510 0x7f77d00c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77d4ea2700 1 -- 172.21.15.33:0/3963380634 >> v1:172.21.15.33:6789/0 conn(0x7f77d00c70c0 legacy=0x7f77d006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77d4ea2700 1 -- 172.21.15.33:0/3963380634 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00c7e30 con 0x7f77d01577a0 2024-05-08T23:32:17.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f821ffff700 1 --2- 172.21.15.33:0/356692155 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 0x7f82200b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77d4ea2700 1 --2- 172.21.15.33:0/3963380634 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77c00088f0 tx=0x7f77c000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ca92d003b14030 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77ceffd700 1 -- 172.21.15.33:0/3963380634 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0010070 con 0x7f77d01577a0 2024-05-08T23:32:17.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77ceffd700 1 -- 172.21.15.33:0/3963380634 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c000da40 con 0x7f77d01577a0 2024-05-08T23:32:17.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f77ceffd700 1 -- 172.21.15.33:0/3963380634 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c000ddb0 con 0x7f77d01577a0 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f821ffff700 1 -- 172.21.15.33:0/356692155 >> v1:172.21.15.42:6789/0 conn(0x7f822015c000 legacy=0x7f82200b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f821ffff700 1 -- 172.21.15.33:0/356692155 >> v1:172.21.15.33:6789/0 conn(0x7f82200b39e0 legacy=0x7f8220169060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f821ffff700 1 -- 172.21.15.33:0/356692155 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822015f2e0 con 0x7f822015e7c0 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f821ffff700 1 --2- 172.21.15.33:0/356692155 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 0x7f82200b4ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82140097f0 tx=0x7f8214009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466551b7321b92b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.879+0000 7f088a25b700 1 -- start start 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08840b3200 0x7f08840cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 0x7f088415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 0x7f088415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884157c00 con 0x7f088415f460 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088415b2d0 con 0x7f08840b3200 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f088a25b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0884157260 con 0x7f088406e220 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f821f7fe700 1 -- 172.21.15.33:0/356692155 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214004230 con 0x7f822015e7c0 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f821f7fe700 1 -- 172.21.15.33:0/356692155 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214004390 con 0x7f822015e7c0 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f821f7fe700 1 -- 172.21.15.33:0/356692155 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140036a0 con 0x7f822015e7c0 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 -- 172.21.15.33:0/3963380634 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 msgr2=0x7f77d0156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 --2- 172.21.15.33:0/3963380634 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77c00088f0 tx=0x7f77c000d330 comp rx=0 tx=0).stop 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f0883fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 0x7f088415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f0883fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 0x7f088415a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53672/0 (socket says 172.21.15.33:53672) 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.880+0000 7f0883fff700 1 -- 172.21.15.33:0/4183728871 learned_addr learned my addr 172.21.15.33:0/4183728871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 0x7f088415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 -- 172.21.15.33:0/3963380634 shutdown_connections 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 --2- 172.21.15.33:0/3963380634 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01577a0 0x7f77d0156db0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 --2- 172.21.15.33:0/3963380634 >> v2:172.21.15.42:3300/0 conn(0x7f77d0150510 0x7f77d00c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 -- 172.21.15.33:0/3963380634 >> 172.21.15.33:0/3963380634 conn(0x7f77d00c8360 msgr2=0x7f77d00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f08837fe700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08840b3200 0x7f08840cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 -- 172.21.15.33:0/3963380634 shutdown_connections 2024-05-08T23:32:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 -- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 msgr2=0x7f088415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 0x7f088415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 -- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08840b3200 msgr2=0x7f08840cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08840b3200 0x7f08840cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f0882ffd700 1 -- 172.21.15.33:0/4183728871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878009550 con 0x7f088415f460 2024-05-08T23:32:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f8227247700 1 -- 172.21.15.33:0/356692155 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 msgr2=0x7f82200b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f8227247700 1 --2- 172.21.15.33:0/356692155 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 0x7f82200b4ab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f82140097f0 tx=0x7f8214009b00 comp rx=0 tx=0).stop 2024-05-08T23:32:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f8227247700 1 -- 172.21.15.33:0/356692155 shutdown_connections 2024-05-08T23:32:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f8227247700 1 --2- 172.21.15.33:0/356692155 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f822015e7c0 0x7f82200b4ab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f8227247700 1 -- 172.21.15.33:0/356692155 >> 172.21.15.33:0/356692155 conn(0x7f82200d0050 msgr2=0x7f82200d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f0882ffd700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 0x7f088415ba80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f087c00c780 tx=0x7f087c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c003e10 con 0x7f088415f460 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c005070 con 0x7f088415f460 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c005420 con 0x7f088415f460 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f8227247700 1 -- 172.21.15.33:0/356692155 shutdown_connections 2024-05-08T23:32:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f8227247700 1 -- 172.21.15.33:0/356692155 wait complete. 2024-05-08T23:32:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f77d6905700 1 -- 172.21.15.33:0/3963380634 wait complete. 2024-05-08T23:32:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f77d6905700 1 Processor -- start 2024-05-08T23:32:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0884157540 con 0x7f088415f460 2024-05-08T23:32:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0884158020 con 0x7f088415f460 2024-05-08T23:32:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f8227247700 1 Processor -- start 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.884+0000 7f77d6905700 1 -- start start 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77d0150510 0x7f77d0144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d01577a0 0x7f77d006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0146500 con 0x7f77d01577a0 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01467c0 con 0x7f77d0150510 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d6905700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c4860 con 0x7f77d01f9f10 2024-05-08T23:32:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d4ea2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f77d4ea2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53694/0 (socket says 172.21.15.33:53694) 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 -- 172.21.15.33:0/987200615 learned_addr learned my addr 172.21.15.33:0/987200615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77cffff700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77d0150510 0x7f77d0144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 -- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77d0150510 msgr2=0x7f77d0144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77d0150510 0x7f77d0144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 -- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d01577a0 msgr2=0x7f77d006d9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d01577a0 0x7f77d006d9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f77d4ea2700 1 -- 172.21.15.33:0/987200615 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00085d0 con 0x7f77d01f9f10 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.885+0000 7f8227247700 1 -- start start 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f822015c000 0x7f8220158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822015e7c0 0x7f82201f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220159bf0 con 0x7f822015c000 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220157530 con 0x7f822015e7c0 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8227247700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822015a3c0 con 0x7f82200b39e0 2024-05-08T23:32:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08640052f0 con 0x7f088415f460 2024-05-08T23:32:17.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8224fe3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8224fe3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53696/0 (socket says 172.21.15.33:53696) 2024-05-08T23:32:17.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f8224fe3700 1 -- 172.21.15.33:0/4182461509 learned_addr learned my addr 172.21.15.33:0/4182461509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f77d4ea2700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d370 tx=0x7f77c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.887+0000 7f82257e4700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822015e7c0 0x7f82201f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f1f1befa700 1 Processor -- start 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.881+0000 7f1f1befa700 1 -- start start 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f1f1befa700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1f140c58c0 0x7f1f14070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f1f1befa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f1f1befa700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f14072e40 con 0x7f1f141670c0 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f1f1befa700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1406b920 con 0x7f1f140c58c0 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.882+0000 7f1f1befa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1406aa30 con 0x7f1f140c9050 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f1f19495700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f1f19495700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53686/0 (socket says 172.21.15.33:53686) 2024-05-08T23:32:17.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.883+0000 7f1f19495700 1 -- 172.21.15.33:0/2557204952 learned_addr learned my addr 172.21.15.33:0/2557204952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 -- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822015e7c0 msgr2=0x7f82201f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822015e7c0 0x7f82201f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 -- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f822015c000 msgr2=0x7f8220158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f822015c000 0x7f8220158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.886+0000 7f1f18c94700 1 -- 172.21.15.33:0/2557204952 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010840119 0 0) 0x7f1f14072e40 con 0x7f1f141670c0 2024-05-08T23:32:17.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f1f18c94700 1 -- 172.21.15.33:0/2557204952 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f000035b0 con 0x7f1f141670c0 2024-05-08T23:32:17.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 -- 172.21.15.33:0/4182461509 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140094a0 con 0x7f82200b39e0 2024-05-08T23:32:17.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f8224fe3700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f821000c730 tx=0x7f821000ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210004c80 con 0x7f82200b39e0 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210004e00 con 0x7f82200b39e0 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210007310 con 0x7f82200b39e0 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f1f19495700 1 -- 172.21.15.33:0/2557204952 >> v2:172.21.15.33:3300/0 conn(0x7f1f140c58c0 msgr2=0x7f1f14070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f1f19495700 1 --2- 172.21.15.33:0/2557204952 >> v2:172.21.15.33:3300/0 conn(0x7f1f140c58c0 0x7f1f14070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f1f19495700 1 -- 172.21.15.33:0/2557204952 >> v1:172.21.15.33:6789/0 conn(0x7f1f141670c0 legacy=0x7f1f14070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f1f19495700 1 -- 172.21.15.33:0/2557204952 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1406ad10 con 0x7f1f140c9050 2024-05-08T23:32:17.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0004130 con 0x7f77d01f9f10 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c0004290 con 0x7f77d01f9f10 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c00036a0 con 0x7f77d01f9f10 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d00c4b40 con 0x7f77d01f9f10 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.888+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00c5100 con 0x7f77d01f9f10 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.889+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8220159e00 con 0x7f82200b39e0 2024-05-08T23:32:17.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.889+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8220156ee0 con 0x7f82200b39e0 2024-05-08T23:32:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.890+0000 7f1f19495700 1 --2- 172.21.15.33:0/2557204952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d1a0 tx=0x7f1f0c00d4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36a5f15cb6b8a2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.891+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f087c005580 con 0x7f088415f460 2024-05-08T23:32:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.891+0000 7f086bfff700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 0x7f087006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f087c088e40 con 0x7f088415f460 2024-05-08T23:32:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f1f18c94700 1 -- 172.21.15.33:0/2557204952 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c014830 con 0x7f1f140c9050 2024-05-08T23:32:17.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f1f18c94700 1 -- 172.21.15.33:0/2557204952 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c014990 con 0x7f1f140c9050 2024-05-08T23:32:17.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f1f18c94700 1 -- 172.21.15.33:0/2557204952 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c014d20 con 0x7f1f140c9050 2024-05-08T23:32:17.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f08837fe700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 0x7f087006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.892+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8210007470 con 0x7f82200b39e0 2024-05-08T23:32:17.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 -- 172.21.15.33:0/2557204952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 msgr2=0x7f1f14070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 --2- 172.21.15.33:0/2557204952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00d1a0 tx=0x7f1f0c00d4b0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f77c0003800 con 0x7f77d01f9f10 2024-05-08T23:32:17.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f08837fe700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 0x7f087006d250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f0878008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 -- 172.21.15.33:0/2557204952 shutdown_connections 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 --2- 172.21.15.33:0/2557204952 >> v2:172.21.15.33:3300/0 conn(0x7f1f140c58c0 0x7f1f14070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 --2- 172.21.15.33:0/2557204952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14070550 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f1f1befa700 1 -- 172.21.15.33:0/2557204952 >> 172.21.15.33:0/2557204952 conn(0x7f1f140c8360 msgr2=0x7f1f140c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f821d7fa700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 0x7f820c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8210089050 con 0x7f82200b39e0 2024-05-08T23:32:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82000052f0 con 0x7f82200b39e0 2024-05-08T23:32:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.893+0000 7f77ccff9700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 0x7f77b406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77c008a070 con 0x7f77d01f9f10 2024-05-08T23:32:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f821ffff700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 0x7f820c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f77cffff700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 0x7f77b406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f1f1befa700 1 -- 172.21.15.33:0/2557204952 shutdown_connections 2024-05-08T23:32:17.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.895+0000 7f77cffff700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 0x7f77b406d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77d014e4a0 tx=0x7f77c8009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.895+0000 7f821ffff700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 0x7f820c06d220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822015ba10 tx=0x7f8214010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.894+0000 7f1f1befa700 1 -- 172.21.15.33:0/2557204952 wait complete. 2024-05-08T23:32:17.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.896+0000 7f1f1befa700 1 Processor -- start 2024-05-08T23:32:17.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.898+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac0052f0 con 0x7f77d01f9f10 2024-05-08T23:32:17.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.898+0000 7ff04dba4700 1 Processor -- start 2024-05-08T23:32:17.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.896+0000 7f1f1befa700 1 -- start start 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f140c58c0 0x7f1f1415e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415f5f0 con 0x7f1f140c58c0 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f141593b0 con 0x7f1f141670c0 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.897+0000 7f1f1befa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1415d8b0 con 0x7f1f140c9050 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.898+0000 7f1f1a497700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.898+0000 7f1f1a497700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54942/0 (socket says 172.21.15.33:54942) 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.898+0000 7f1f1a497700 1 -- 172.21.15.33:0/1304447319 learned_addr learned my addr 172.21.15.33:0/1304447319 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 -- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 msgr2=0x7f1f14158e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14158e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 -- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f140c58c0 msgr2=0x7f1f1415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f140c58c0 0x7f1f1415e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 -- 172.21.15.33:0/1304447319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c007b10 con 0x7f1f141670c0 2024-05-08T23:32:17.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f1a497700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c500 tx=0x7f1f0400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f087c053f70 con 0x7f088415f460 2024-05-08T23:32:17.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7ff04dba4700 1 -- start start 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040052d0 con 0x7f1f141670c0 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04005430 con 0x7f1f141670c0 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.899+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040038a0 con 0x7f1f141670c0 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1415e150 con 0x7f1f141670c0 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14157f20 con 0x7f1f141670c0 2024-05-08T23:32:17.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f82100542a0 con 0x7f82200b39e0 2024-05-08T23:32:17.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.901+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14075460 con 0x7f1f141670c0 2024-05-08T23:32:17.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.901+0000 7f8e8de2a700 1 Processor -- start 2024-05-08T23:32:17.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.901+0000 7f8e8de2a700 1 -- start start 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1f04003a00 con 0x7f1f141670c0 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff04dba4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0480c68e0 0x7ff048071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff04dba4700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0481582e0 con 0x7ff0480c6e30 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff04dba4700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff048157e50 con 0x7ff0480c6310 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff04dba4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff048157310 con 0x7ff0480c68e0 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff047fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff0480c6e30 0x7ff04806a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52022/0 (socket says 172.21.15.33:52022) 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.900+0000 7ff047fff700 1 -- 172.21.15.33:0/2663802107 learned_addr learned my addr 172.21.15.33:0/2663802107 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:17.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.901+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594156187 0 0) 0x7ff0481582e0 con 0x7ff0480c6e30 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.901+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03000bdc0 con 0x7ff0480c6e30 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1047544149 0 0) 0x7ff03000bdc0 con 0x7ff0480c6e30 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0480c68e0 msgr2=0x7ff048071710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0480c68e0 0x7ff048071710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 >> v1:172.21.15.42:6789/0 conn(0x7ff0480c6310 legacy=0x7ff04806baa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480c66f0 con 0x7ff0480c6e30 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7ff0400019f0 con 0x7ff0480c6e30 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 >> v1:172.21.15.33:6789/0 conn(0x7ff0480c6e30 legacy=0x7ff04806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000c240 0x7ff03000c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff03000d060 0x7ff03000d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff03000da70 0x7ff0300094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048157310 con 0x7ff03000d060 2024-05-08T23:32:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030009c30 con 0x7ff03000da70 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff030009df0 con 0x7ff03000c240 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.902+0000 7ff047fff700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff03000da70 0x7ff0300094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e8de2a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8e88158ce0 0x7f8e8806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e8de2a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8e880c9220 0x7f8e880719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e8de2a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e880ca1d0 con 0x7f8e8815f230 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e8de2a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8806aed0 con 0x7f8e880c9220 2024-05-08T23:32:17.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e8de2a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8806b9a0 con 0x7f8e88158ce0 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f1f0a7fc700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 0x7f1ef806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f040887b0 con 0x7f1f141670c0 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7ff0477fe700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000c240 0x7ff03000c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8e88158ce0 0x7f8e8806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e86ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8e880c9220 0x7f8e880719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8e88158ce0 0x7f8e8806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54960/0 (socket says 172.21.15.33:54960) 2024-05-08T23:32:17.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 -- 172.21.15.33:0/3389277130 learned_addr learned my addr 172.21.15.33:0/3389277130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7ff046ffd700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff03000d060 0x7ff03000d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 -- 172.21.15.33:0/3389277130 >> v2:172.21.15.33:3300/0 conn(0x7f8e880c9220 msgr2=0x7f8e880719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 --2- 172.21.15.33:0/3389277130 >> v2:172.21.15.33:3300/0 conn(0x7f8e880c9220 0x7f8e880719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.903+0000 7f8e877fe700 1 -- 172.21.15.33:0/3389277130 >> v1:172.21.15.42:6789/0 conn(0x7f8e8815f230 legacy=0x7f8e88072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7f8e877fe700 1 -- 172.21.15.33:0/3389277130 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8806c540 con 0x7f8e88158ce0 2024-05-08T23:32:17.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7f8e877fe700 1 --2- 172.21.15.33:0/3389277130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e88158ce0 0x7f8e8806a990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806fcb0 tx=0x7f8e78009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc8c9bff947c38a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7f1f19c96700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 0x7f1ef806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7f1f19c96700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 0x7f1ef806d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009710 tx=0x7f1f10006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7ff046ffd700 1 -- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000c240 msgr2=0x7ff03000c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7ff046ffd700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000c240 0x7ff03000c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.904+0000 7ff046ffd700 1 -- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff03000da70 msgr2=0x7ff0300094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff046ffd700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff03000da70 0x7ff0300094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff046ffd700 1 -- 172.21.15.33:0/2663802107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480c66f0 con 0x7ff03000d060 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff046ffd700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff03000d060 0x7ff03000d4e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ca290 tx=0x7ff03000e090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06bad7f78ebdc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011420 con 0x7ff03000d060 2024-05-08T23:32:17.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c0115a0 con 0x7ff03000d060 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.905+0000 7ff0467fc700 1 -- 172.21.15.33:0/2663802107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011950 con 0x7ff03000d060 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.906+0000 7ff04dba4700 1 -- 172.21.15.33:0/2663802107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff03000d060 msgr2=0x7ff03000d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.906+0000 7ff04dba4700 1 --2- 172.21.15.33:0/2663802107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff03000d060 0x7ff03000d4e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ca290 tx=0x7ff03000e090 comp rx=0 tx=0).stop 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.906+0000 7ff047fff700 1 -- 172.21.15.33:0/2663802107 reap_dead start 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.906+0000 7ff04dba4700 1 -- 172.21.15.33:0/2663802107 shutdown_connections 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.906+0000 7ff04dba4700 1 -- 172.21.15.33:0/2663802107 >> 172.21.15.33:0/2663802107 conn(0x7ff04806d2b0 msgr2=0x7ff04806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.907+0000 7ff04dba4700 1 -- 172.21.15.33:0/2663802107 shutdown_connections 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.907+0000 7ff04dba4700 1 -- 172.21.15.33:0/2663802107 wait complete. 2024-05-08T23:32:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.908+0000 7ff04dba4700 1 Processor -- start 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.909+0000 7f8e867fc700 1 -- 172.21.15.33:0/3389277130 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800b870 con 0x7f8e88158ce0 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.909+0000 7f8e867fc700 1 -- 172.21.15.33:0/3389277130 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7800b9d0 con 0x7f8e88158ce0 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.909+0000 7f8e867fc700 1 -- 172.21.15.33:0/3389277130 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800bd60 con 0x7f8e88158ce0 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7f8e8de2a700 1 -- 172.21.15.33:0/3389277130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e88158ce0 msgr2=0x7f8e8806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/3389277130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e88158ce0 0x7f8e8806a990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806fcb0 tx=0x7f8e78009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f77c00552c0 con 0x7f77d01f9f10 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.909+0000 7ff04dba4700 1 -- start start 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000d060 0x7ff0481e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff04806c480 0x7ff0480c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480c1f80 con 0x7ff0480c1090 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480c57c0 con 0x7ff04806c480 2024-05-08T23:32:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff04dba4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0481ef510 con 0x7ff03000d060 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff047fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff047fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52978/0 (socket says 172.21.15.33:52978) 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff047fff700 1 -- 172.21.15.33:0/215530637 learned_addr learned my addr 172.21.15.33:0/215530637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.910+0000 7ff0477fe700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000d060 0x7ff0481e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 -- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000d060 msgr2=0x7ff0481e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000d060 0x7ff0481e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 -- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff04806c480 msgr2=0x7ff0480c0b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff04806c480 0x7ff0480c0b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 -- 172.21.15.33:0/215530637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480c4de0 con 0x7ff0480c1090 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff047fff700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff04806e850 tx=0x7ff040007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.912+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1f04053910 con 0x7f1f141670c0 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000ecd0 con 0x7ff0480c1090 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000ee30 con 0x7ff0480c1090 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.911+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040035650 con 0x7ff0480c1090 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.912+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0480c5080 con 0x7ff0480c1090 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.913+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481e1d20 con 0x7ff0480c1090 2024-05-08T23:32:17.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.915+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff0400357b0 con 0x7ff0480c1090 2024-05-08T23:32:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7ff02ffff700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 0x7ff03406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0400ab9a0 con 0x7ff0480c1090 2024-05-08T23:32:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7f8e8de2a700 1 -- 172.21.15.33:0/3389277130 shutdown_connections 2024-05-08T23:32:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/3389277130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e88158ce0 0x7f8e8806a990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/3389277130 >> v2:172.21.15.33:3300/0 conn(0x7f8e880c9220 0x7f8e880719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.916+0000 7f8e8de2a700 1 -- 172.21.15.33:0/3389277130 >> 172.21.15.33:0/3389277130 conn(0x7f8e8806d2b0 msgr2=0x7f8e8806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.917+0000 7f8e8de2a700 1 -- 172.21.15.33:0/3389277130 shutdown_connections 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.917+0000 7f8e8de2a700 1 -- 172.21.15.33:0/3389277130 wait complete. 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.919+0000 7f8e8de2a700 1 Processor -- start 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.919+0000 7f8e8de2a700 1 -- start start 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8e880c9220 0x7f8e88159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e8815f230 0x7f8e881ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881fc940 con 0x7f8e88158ce0 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e88155e80 con 0x7f8e8815f230 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.920+0000 7f8e8de2a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881fd160 con 0x7f8e880c9220 2024-05-08T23:32:17.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.922+0000 7ff0477fe700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 0x7ff03406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.923+0000 7f8e86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.923+0000 7f8e86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52988/0 (socket says 172.21.15.33:52988) 2024-05-08T23:32:17.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.923+0000 7f8e86ffd700 1 -- 172.21.15.33:0/822136121 learned_addr learned my addr 172.21.15.33:0/822136121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:17.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.923+0000 7ff0477fe700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 0x7ff03406d180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff038009a90 tx=0x7ff038006f70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 -- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8e880c9220 msgr2=0x7f8e88159b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8e880c9220 0x7f8e88159b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 -- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e8815f230 msgr2=0x7f8e881ffeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e8815f230 0x7f8e881ffeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 -- 172.21.15.33:0/822136121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009550 con 0x7f8e88158ce0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e86ffd700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e840 tx=0x7f8e7c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.924+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c013610 con 0x7f8e88158ce0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.925+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c013770 con 0x7f8e88158ce0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.925+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c013b00 con 0x7f8e88158ce0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.925+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e882003f0 con 0x7f8e88158ce0 2024-05-08T23:32:17.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.925+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e881558c0 con 0x7f8e88158ce0 2024-05-08T23:32:17.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.926+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e88075460 con 0x7f8e88158ce0 2024-05-08T23:32:17.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.934+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0480c5970 con 0x7ff0480c1090 2024-05-08T23:32:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.947+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff0480c5970 con 0x7ff0480c1090 2024-05-08T23:32:17.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.937+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8e7c013c60 con 0x7f8e88158ce0 2024-05-08T23:32:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.939+0000 7f8e6ffff700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 0x7f8e7006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.939+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e7c089490 con 0x7f8e88158ce0 2024-05-08T23:32:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.939+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8e7c089870 con 0x7f8e88158ce0 2024-05-08T23:32:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.940+0000 7f8e877fe700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 0x7f8e7006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:17.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:17.941+0000 7f8e877fe700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 0x7f8e7006d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009c10 tx=0x7f8e78008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.094+0000 7f00e829f700 1 Processor -- start 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.094+0000 7f00e829f700 1 -- start start 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.095+0000 7f00e829f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00e00c2770 0x7f00e00c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.095+0000 7f00e829f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.095+0000 7f00e829f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e006ba40 con 0x7f00e01589d0 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.095+0000 7f00e829f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e015f180 con 0x7f00e00c2770 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.095+0000 7f00e829f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e00c8210 con 0x7f00e00c7d80 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.096+0000 7f00e683c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.096+0000 7f00e683c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53744/0 (socket says 172.21.15.33:53744) 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 -- 172.21.15.33:0/1353301712 learned_addr learned my addr 172.21.15.33:0/1353301712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 -- 172.21.15.33:0/1353301712 >> v2:172.21.15.33:3300/0 conn(0x7f00e00c2770 msgr2=0x7f00e00c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 --2- 172.21.15.33:0/1353301712 >> v2:172.21.15.33:3300/0 conn(0x7f00e00c2770 0x7f00e00c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 -- 172.21.15.33:0/1353301712 >> v1:172.21.15.33:6789/0 conn(0x7f00e01589d0 legacy=0x7f00e006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 -- 172.21.15.33:0/1353301712 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0070930 con 0x7f00e00c7d80 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.097+0000 7f00e683c700 1 --2- 172.21.15.33:0/1353301712 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0098f0 tx=0x7f00dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d570a45c5a8d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.098+0000 7f00e5039700 1 -- 172.21.15.33:0/1353301712 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00b8a0 con 0x7f00e00c7d80 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.098+0000 7f00e5039700 1 -- 172.21.15.33:0/1353301712 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc00ba00 con 0x7f00e00c7d80 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.098+0000 7f00e5039700 1 -- 172.21.15.33:0/1353301712 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00bd90 con 0x7f00e00c7d80 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.099+0000 7f00e829f700 1 -- 172.21.15.33:0/1353301712 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 msgr2=0x7f00e0070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.099+0000 7f00e829f700 1 --2- 172.21.15.33:0/1353301712 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0098f0 tx=0x7f00dc009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 -- 172.21.15.33:0/1353301712 shutdown_connections 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 --2- 172.21.15.33:0/1353301712 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e00c7d80 0x7f00e0070dc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 --2- 172.21.15.33:0/1353301712 >> v2:172.21.15.33:3300/0 conn(0x7f00e00c2770 0x7f00e00c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 -- 172.21.15.33:0/1353301712 >> 172.21.15.33:0/1353301712 conn(0x7f00e00c8360 msgr2=0x7f00e00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 -- 172.21.15.33:0/1353301712 shutdown_connections 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.100+0000 7f00e829f700 1 -- 172.21.15.33:0/1353301712 wait complete. 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.102+0000 7f00e829f700 1 Processor -- start 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.103+0000 7f00e829f700 1 -- start start 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00e00c2770 0x7f00e013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e01589d0 0x7f00e013b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0149db0 con 0x7f00e00c7d80 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e013b360 con 0x7f00e00c2770 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e829f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e013d100 con 0x7f00e01589d0 2024-05-08T23:32:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e583a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e583a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53006/0 (socket says 172.21.15.33:53006) 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.104+0000 7f00e583a700 1 -- 172.21.15.33:0/1346385157 learned_addr learned my addr 172.21.15.33:0/1346385157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e603b700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00e00c2770 0x7f00e013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e683c700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e01589d0 0x7f00e013b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 -- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e01589d0 msgr2=0x7f00e013b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e01589d0 0x7f00e013b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 -- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00e00c2770 msgr2=0x7f00e013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00e00c2770 0x7f00e013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 -- 172.21.15.33:0/1346385157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc009550 con 0x7f00e00c7d80 2024-05-08T23:32:18.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.105+0000 7f00e583a700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00d800ba50 tx=0x7f00d800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:18.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.108+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800c930 con 0x7f00e00c7d80 2024-05-08T23:32:18.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.108+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800cab0 con 0x7f00e00c7d80 2024-05-08T23:32:18.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.108+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d800ce60 con 0x7f00e00c7d80 2024-05-08T23:32:18.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.108+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e014a000 con 0x7f00e00c7d80 2024-05-08T23:32:18.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.108+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e013aff0 con 0x7f00e00c7d80 2024-05-08T23:32:18.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.113+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f00d801d3b0 con 0x7f00e00c7d80 2024-05-08T23:32:18.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.114+0000 7f00d6ffd700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 0x7f00c406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:18.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.114+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00d8089080 con 0x7f00e00c7d80 2024-05-08T23:32:18.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.115+0000 7f00e603b700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 0x7f00c406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:18.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.116+0000 7f00e603b700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 0x7f00c406d220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00d0007fd0 tx=0x7f00d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:18.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.116+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c80052f0 con 0x7f00e00c7d80 2024-05-08T23:32:18.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.126+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f00d8019080 con 0x7f00e00c7d80 2024-05-08T23:32:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.308+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2afc161760 con 0x7f2afc161c70 2024-05-08T23:32:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.309+0000 7f2af2ffd700 1 -- 172.21.15.33:0/3987086458 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2aec057a00 con 0x7f2afc161c70 2024-05-08T23:32:18.312 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 msgr2=0x7f2ae006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 0x7f2ae006d1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2af80013a0 tx=0x7f2af8001330 comp rx=0 tx=0).stop 2024-05-08T23:32:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 msgr2=0x7f2afc142e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 0x7f2afc142e30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b3c0 tx=0x7f2aec00b6d0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 shutdown_connections 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2ae006ace0 0x7f2ae006d1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2afc161c70 0x7f2afc142e30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2afc0722c0 0x7f2afc146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 --2- 172.21.15.33:0/3987086458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2afc0c70d0 0x7f2afc141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.313+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 >> 172.21.15.33:0/3987086458 conn(0x7f2afc0c89c0 msgr2=0x7f2afc0c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.314+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 shutdown_connections 2024-05-08T23:32:18.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.314+0000 7f2b044d3700 1 -- 172.21.15.33:0/3987086458 wait complete. 2024-05-08T23:32:18.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-05-08T23:32:18.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.353+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f77ac0055a0 con 0x7f77d01f9f10 2024-05-08T23:32:18.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.355+0000 7f77ccff9700 1 -- 172.21.15.33:0/987200615 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77c001b020 con 0x7f77d01f9f10 2024-05-08T23:32:18.359 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 msgr2=0x7f77b406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 0x7f77b406d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77d014e4a0 tx=0x7f77c8009830 comp rx=0 tx=0).stop 2024-05-08T23:32:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 msgr2=0x7f77d01fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77c000d370 tx=0x7f77c0004750 comp rx=0 tx=0).stop 2024-05-08T23:32:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f82000055a0 con 0x7f82200b39e0 2024-05-08T23:32:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 shutdown_connections 2024-05-08T23:32:18.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f77b406ad60 0x7f77b406d220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f77d01f9f10 0x7f77d01fa3a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77d0150510 0x7f77d0144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 --2- 172.21.15.33:0/987200615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d01577a0 0x7f77d006d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 >> 172.21.15.33:0/987200615 conn(0x7f77d00c8360 msgr2=0x7f77d014e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.362+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 shutdown_connections 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.363+0000 7f77d6905700 1 -- 172.21.15.33:0/987200615 wait complete. 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.363+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1f14158200 con 0x7f1f141670c0 2024-05-08T23:32:18.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.364+0000 7f821d7fa700 1 -- 172.21.15.33:0/4182461509 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82100578c0 con 0x7f82200b39e0 2024-05-08T23:32:18.365 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.365+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0864005870 con 0x7f088415f460 2024-05-08T23:32:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.367+0000 7f1f0a7fc700 1 -- 172.21.15.33:0/1304447319 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f04017090 con 0x7f1f141670c0 2024-05-08T23:32:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.367+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 msgr2=0x7f820c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.367+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 0x7f820c06d220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822015ba10 tx=0x7f8214010000 comp rx=0 tx=0).stop 2024-05-08T23:32:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.367+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 msgr2=0x7f82200c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.367+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f821000c730 tx=0x7f821000ca40 comp rx=0 tx=0).stop 2024-05-08T23:32:18.368 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f086bfff700 1 -- 172.21.15.33:0/4183728871 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f087c017070 con 0x7f088415f460 2024-05-08T23:32:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 shutdown_connections 2024-05-08T23:32:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f820c06ad60 0x7f820c06d220 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822015e7c0 0x7f82201f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f82200b39e0 0x7f82200c5f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 --2- 172.21.15.33:0/4182461509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f822015c000 0x7f8220158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.368+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 >> 172.21.15.33:0/4182461509 conn(0x7f82200d0050 msgr2=0x7f82200cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 msgr2=0x7f1ef806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 0x7f1ef806d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009710 tx=0x7f1f10006c80 comp rx=0 tx=0).stop 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 msgr2=0x7f1f1420e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c500 tx=0x7f1f0400c8c0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.372 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 shutdown_connections 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.371+0000 7f8227247700 1 -- 172.21.15.33:0/4182461509 wait complete. 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 shutdown_connections 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ef806ac90 0x7f1ef806d150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f141670c0 0x7f1f1420e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f140c58c0 0x7f1f1415e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 --2- 172.21.15.33:0/1304447319 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1f140c9050 0x7f1f14158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.373+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 >> 172.21.15.33:0/1304447319 conn(0x7f1f140c8360 msgr2=0x7f1f140caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 msgr2=0x7f087006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 0x7f087006d250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f0878008030 comp rx=0 tx=0).stop 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 msgr2=0x7f088415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 0x7f088415ba80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f087c00c780 tx=0x7f087c00ca90 comp rx=0 tx=0).stop 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 shutdown_connections 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f087006ad90 0x7f087006d250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f088406e220 0x7f088415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08840b3200 0x7f08840cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 --2- 172.21.15.33:0/4183728871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f088415f460 0x7f088415ba80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.376+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 >> 172.21.15.33:0/4183728871 conn(0x7f088406cd80 msgr2=0x7f088406d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 shutdown_connections 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.375+0000 7f1f1befa700 1 -- 172.21.15.33:0/1304447319 wait complete. 2024-05-08T23:32:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.377+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 shutdown_connections 2024-05-08T23:32:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.377+0000 7f088a25b700 1 -- 172.21.15.33:0/4183728871 wait complete. 2024-05-08T23:32:18.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.379+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8e880c6250 con 0x7f8e88158ce0 2024-05-08T23:32:18.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-05-08T23:32:18.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.387+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0480c5970 con 0x7ff0480c1090 2024-05-08T23:32:18.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.387+0000 7f8e6ffff700 1 -- 172.21.15.33:0/822136121 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e7c054650 con 0x7f8e88158ce0 2024-05-08T23:32:18.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.388+0000 7ff02ffff700 1 -- 172.21.15.33:0/215530637 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff0480c5970 con 0x7ff0480c1090 2024-05-08T23:32:18.388 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:18.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.390+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 msgr2=0x7f8e7006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.390+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 0x7f8e7006d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009c10 tx=0x7f8e78008030 comp rx=0 tx=0).stop 2024-05-08T23:32:18.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.390+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 msgr2=0x7f8e88155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.390+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e840 tx=0x7f8e7c00ec00 comp rx=0 tx=0).stop 2024-05-08T23:32:18.391 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.392+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 msgr2=0x7ff03406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.392+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 0x7ff03406d180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff038009a90 tx=0x7ff038006f70 comp rx=0 tx=0).stop 2024-05-08T23:32:18.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.392+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 msgr2=0x7ff04814be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.392+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff04806e850 tx=0x7ff040007c90 comp rx=0 tx=0).stop 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 shutdown_connections 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff03406acc0 0x7ff03406d180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0480c1090 0x7ff04814be80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff03000d060 0x7ff0481e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 --2- 172.21.15.33:0/215530637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff04806c480 0x7ff0480c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 >> 172.21.15.33:0/215530637 conn(0x7ff04806d2b0 msgr2=0x7ff048156330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 shutdown_connections 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8e7006af20 0x7f8e7006d3e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e8815f230 0x7f8e881ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8e880c9220 0x7f8e88159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 --2- 172.21.15.33:0/822136121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e88158ce0 0x7f8e88155210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 >> 172.21.15.33:0/822136121 conn(0x7f8e8806d2b0 msgr2=0x7f8e880c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 shutdown_connections 2024-05-08T23:32:18.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.393+0000 7f8e8de2a700 1 -- 172.21.15.33:0/822136121 wait complete. 2024-05-08T23:32:18.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-05-08T23:32:18.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.398+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 shutdown_connections 2024-05-08T23:32:18.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.399+0000 7ff04dba4700 1 -- 172.21.15.33:0/215530637 wait complete. 2024-05-08T23:32:18.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-05-08T23:32:18.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-05-08T23:32:18.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-08T23:32:18.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-05-08T23:32:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.488+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f00c8005160 con 0x7f00e00c7d80 2024-05-08T23:32:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.488+0000 7f00d6ffd700 1 -- 172.21.15.33:0/1346385157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f00d80541b0 con 0x7f00e00c7d80 2024-05-08T23:32:18.489 INFO:teuthology.orchestra.run.smithi033.stdout:55834574852 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 msgr2=0x7f00c406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 0x7f00c406d220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00d0007fd0 tx=0x7f00d0009040 comp rx=0 tx=0).stop 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 msgr2=0x7f00e0141500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f00d800ba50 tx=0x7f00d800bd60 comp rx=0 tx=0).stop 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 shutdown_connections 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f00c406ad60 0x7f00c406d220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f00e01589d0 0x7f00e013b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00e00c2770 0x7f00e013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 --2- 172.21.15.33:0/1346385157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e00c7d80 0x7f00e0141500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:18.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 >> 172.21.15.33:0/1346385157 conn(0x7f00e00c8360 msgr2=0x7f00e0157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:18.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.490+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 shutdown_connections 2024-05-08T23:32:18.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:18.491+0000 7f00e829f700 1 -- 172.21.15.33:0/1346385157 wait complete. 2024-05-08T23:32:18.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-05-08T23:32:19.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T23:32:19.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T23:32:19.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T23:32:19.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T23:32:19.403 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T23:32:19.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T23:32:19.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T23:32:19.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T23:32:19.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.530+0000 7f81aa244700 1 Processor -- start 2024-05-08T23:32:19.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.530+0000 7f81aa244700 1 -- start start 2024-05-08T23:32:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81aa244700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f81a40c6310 0x7f81a406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81aa244700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a4161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81aa244700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a41582e0 con 0x7f81a40c68e0 2024-05-08T23:32:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81aa244700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a4157310 con 0x7f81a40c6310 2024-05-08T23:32:19.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81aa244700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4157e50 con 0x7f81a4071e20 2024-05-08T23:32:19.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81a2ffd700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f81a40c68e0 0x7f81a4071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59036/0 (socket says 172.21.15.33:59036) 2024-05-08T23:32:19.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81a2ffd700 1 -- 172.21.15.33:0/3197457736 learned_addr learned my addr 172.21.15.33:0/3197457736 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:19.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.532+0000 7f81a3fff700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a4161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.533+0000 7f81a37fe700 1 --2- 172.21.15.33:0/3197457736 >> v2:172.21.15.42:3300/0 conn(0x7f81a40c6310 0x7f81a406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.534+0000 7f81a37fe700 1 -- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 msgr2=0x7f81a4161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.534+0000 7f81a37fe700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a4161bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.534+0000 7f81a37fe700 1 -- 172.21.15.33:0/3197457736 >> v1:172.21.15.42:6789/0 conn(0x7f81a40c68e0 legacy=0x7f81a4071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.534+0000 7f81a37fe700 1 -- 172.21.15.33:0/3197457736 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40c66f0 con 0x7f81a40c6310 2024-05-08T23:32:19.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.536+0000 7f81a37fe700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c6310 0x7f81a406baa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8198009840 tx=0x7f8198009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1829e113ea48a638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.536+0000 7f81a27fc700 1 -- 172.21.15.33:0/3197457736 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198004230 con 0x7f81a40c6310 2024-05-08T23:32:19.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.537+0000 7f81a27fc700 1 -- 172.21.15.33:0/3197457736 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8198004390 con 0x7f81a40c6310 2024-05-08T23:32:19.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.537+0000 7f81a27fc700 1 -- 172.21.15.33:0/3197457736 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980036a0 con 0x7f81a40c6310 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 -- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c6310 msgr2=0x7f81a406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c6310 0x7f81a406baa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8198009840 tx=0x7f8198009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 -- 172.21.15.33:0/3197457736 shutdown_connections 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a4161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 --2- 172.21.15.33:0/3197457736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c6310 0x7f81a406baa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.538+0000 7f81aa244700 1 -- 172.21.15.33:0/3197457736 >> 172.21.15.33:0/3197457736 conn(0x7f81a406d2b0 msgr2=0x7f81a406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.539+0000 7f81aa244700 1 -- 172.21.15.33:0/3197457736 shutdown_connections 2024-05-08T23:32:19.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.539+0000 7f81aa244700 1 -- 172.21.15.33:0/3197457736 wait complete. 2024-05-08T23:32:19.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.541+0000 7f81aa244700 1 Processor -- start 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.542+0000 7f81aa244700 1 -- start start 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c68e0 0x7f81a41f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41fcb00 con 0x7f81a40c6310 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a41533e0 con 0x7f81a40c68e0 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.543+0000 7f81aa244700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c34c0 con 0x7f81a4071e20 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53010/0 (socket says 172.21.15.33:53010) 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 -- 172.21.15.33:0/604837834 learned_addr learned my addr 172.21.15.33:0/604837834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 -- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 msgr2=0x7f81a415a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a415a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 -- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c68e0 msgr2=0x7f81a41f6e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:19.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c68e0 0x7f81a41f6e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.544+0000 7f81a2ffd700 1 -- 172.21.15.33:0/604837834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81980094a0 con 0x7f81a40c6310 2024-05-08T23:32:19.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.545+0000 7f81a2ffd700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f819400b580 tx=0x7f819400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.546+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194011830 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.546+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81940119b0 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.546+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194011d60 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.546+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a40c37a0 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.546+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a40c3d60 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.549+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f819401a400 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.550+0000 7f818bfff700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 0x7f819006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.550+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8194089350 con 0x7f81a40c6310 2024-05-08T23:32:19.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.550+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4075460 con 0x7f81a40c6310 2024-05-08T23:32:19.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.551+0000 7f81a37fe700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 0x7f819006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.551+0000 7f81a37fe700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 0x7f819006d250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81980090b0 tx=0x7f8198012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.560+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8194015080 con 0x7f81a40c6310 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.595+0000 7fb3193fd700 1 Processor -- start 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.595+0000 7fb3193fd700 1 -- start start 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.597+0000 7fb3193fd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb3140c70c0 0x7fb3140c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.597+0000 7fb3193fd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb314157c30 con 0x7fb3141577a0 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.597+0000 7fb3193fd700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140c8110 con 0x7fb314150510 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.597+0000 7fb3193fd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140c7e30 con 0x7fb3140c70c0 2024-05-08T23:32:19.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.598+0000 7fb312ffd700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fb314150510 0x7fb31406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59050/0 (socket says 172.21.15.33:59050) 2024-05-08T23:32:19.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.598+0000 7fb312ffd700 1 -- 172.21.15.33:0/3064484594 learned_addr learned my addr 172.21.15.33:0/3064484594 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:19.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.598+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233257936 0 0) 0x7fb314157c30 con 0x7fb3141577a0 2024-05-08T23:32:19.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc0035b0 con 0x7fb3141577a0 2024-05-08T23:32:19.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119764399 0 0) 0x7fb2fc0035b0 con 0x7fb3141577a0 2024-05-08T23:32:19.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 >> v2:172.21.15.33:3300/0 conn(0x7fb3140c70c0 msgr2=0x7fb3140c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 --2- 172.21.15.33:0/3064484594 >> v2:172.21.15.33:3300/0 conn(0x7fb3140c70c0 0x7fb3140c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 >> v1:172.21.15.42:6789/0 conn(0x7fb314150510 legacy=0x7fb31406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140c6300 con 0x7fb3141577a0 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fb308001670 con 0x7fb3141577a0 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 >> v1:172.21.15.33:6789/0 conn(0x7fb3141577a0 legacy=0x7fb314160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 0x7fb2fc003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc004180 0x7fb2fc00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb2fc00b690 0x7fb2fc00db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c7e30 con 0x7fb2fc0037f0 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00e480 con 0x7fb2fc004180 2024-05-08T23:32:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.599+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00e640 con 0x7fb2fc00b690 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.600+0000 7fb3127fc700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc004180 0x7fb2fc00b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.600+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 0x7fb2fc003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.600+0000 7fb3137fe700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb2fc00b690 0x7fb2fc00db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 -- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb2fc00b690 msgr2=0x7fb2fc00db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb2fc00b690 0x7fb2fc00db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 -- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc004180 msgr2=0x7fb2fc00b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc004180 0x7fb2fc00b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 -- 172.21.15.33:0/3064484594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140c6300 con 0x7fb2fc0037f0 2024-05-08T23:32:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.601+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 0x7fb2fc003bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0057d0 tx=0x7fb2fc004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad4687d92df1c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.602+0000 7fb3137fe700 1 -- 172.21.15.33:0/3064484594 reap_dead start 2024-05-08T23:32:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.602+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00cbd0 con 0x7fb2fc0037f0 2024-05-08T23:32:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.602+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00cd30 con 0x7fb2fc0037f0 2024-05-08T23:32:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.602+0000 7fb311ffb700 1 -- 172.21.15.33:0/3064484594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00cea0 con 0x7fb2fc0037f0 2024-05-08T23:32:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.603+0000 7fb3193fd700 1 -- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 msgr2=0x7fb2fc003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.603+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 0x7fb2fc003bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0057d0 tx=0x7fb2fc004870 comp rx=0 tx=0).stop 2024-05-08T23:32:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.604+0000 7fb3193fd700 1 -- 172.21.15.33:0/3064484594 shutdown_connections 2024-05-08T23:32:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.604+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3064484594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2fc0037f0 0x7fb2fc003bf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.604+0000 7fb3193fd700 1 -- 172.21.15.33:0/3064484594 >> 172.21.15.33:0/3064484594 conn(0x7fb3140c8360 msgr2=0x7fb3140c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.604+0000 7fb3193fd700 1 -- 172.21.15.33:0/3064484594 shutdown_connections 2024-05-08T23:32:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.604+0000 7fb3193fd700 1 -- 172.21.15.33:0/3064484594 wait complete. 2024-05-08T23:32:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.607+0000 7fb3193fd700 1 Processor -- start 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.607+0000 7fb3193fd700 1 -- start start 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.608+0000 7fb3193fd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0037f0 0x7fb314153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3193fd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb31414e0c0 0x7fb31414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3193fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3193fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31414ea80 con 0x7fb314154160 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3193fd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314154b30 con 0x7fb2fc0037f0 2024-05-08T23:32:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3193fd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb314200220 con 0x7fb31414e0c0 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53028/0 (socket says 172.21.15.33:53028) 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.609+0000 7fb3137fe700 1 -- 172.21.15.33:0/3601342446 learned_addr learned my addr 172.21.15.33:0/3601342446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 -- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb31414e0c0 msgr2=0x7fb31414e540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb31414e0c0 0x7fb31414e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 -- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0037f0 msgr2=0x7fb314153560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0037f0 0x7fb314153560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 -- 172.21.15.33:0/3601342446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314152500 con 0x7fb314154160 2024-05-08T23:32:19.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.610+0000 7fb3137fe700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb31415a940 tx=0x7fb30800eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.612+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080053a0 con 0x7fb314154160 2024-05-08T23:32:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.612+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308005520 con 0x7fb314154160 2024-05-08T23:32:19.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.612+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080038c0 con 0x7fb314154160 2024-05-08T23:32:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.613+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3141527a0 con 0x7fb314154160 2024-05-08T23:32:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.613+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb314152d60 con 0x7fb314154160 2024-05-08T23:32:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.617+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb308003a20 con 0x7fb314154160 2024-05-08T23:32:19.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.617+0000 7fb3037fe700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 0x7fb2f806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.617+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb308088b60 con 0x7fb314154160 2024-05-08T23:32:19.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.618+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 0x7fb2f806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.619+0000 7fb312ffd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 0x7fb2f806d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb30c018000 tx=0x7fb30c013000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.619+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f00052f0 con 0x7fb314154160 2024-05-08T23:32:19.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.634+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb308019080 con 0x7fb314154160 2024-05-08T23:32:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.657+0000 7fb1abe44700 1 Processor -- start 2024-05-08T23:32:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.658+0000 7fb1abe44700 1 -- start start 2024-05-08T23:32:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.659+0000 7fb1abe44700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb1a40c70c0 0x7fb1a40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.659+0000 7fb1abe44700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.659+0000 7fb1abe44700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a41572f0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.659+0000 7fb1abe44700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a4157c30 con 0x7fb1a40c70c0 2024-05-08T23:32:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.659+0000 7fb1abe44700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40c7e30 con 0x7fb1a41577a0 2024-05-08T23:32:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1aa3e1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7f835a519700 1 Processor -- start 2024-05-08T23:32:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1a93df700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb1a40c70c0 0x7fb1a40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1aa3e1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53800/0 (socket says 172.21.15.33:53800) 2024-05-08T23:32:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1aa3e1700 1 -- 172.21.15.33:0/2327363382 learned_addr learned my addr 172.21.15.33:0/2327363382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1a93df700 1 -- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 msgr2=0x7fb1a4156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1a93df700 1 --2- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1a93df700 1 -- 172.21.15.33:0/2327363382 >> v1:172.21.15.33:6789/0 conn(0x7fb1a40c8e50 legacy=0x7fb1a41502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7fb1a93df700 1 -- 172.21.15.33:0/2327363382 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40c6300 con 0x7fb1a40c70c0 2024-05-08T23:32:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.661+0000 7f835a519700 1 -- start start 2024-05-08T23:32:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7fb1a93df700 1 --2- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c70c0 0x7fb1a40c7780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb194009840 tx=0x7fb194009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a9ece18f59453c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7fb1a8bde700 1 -- 172.21.15.33:0/2327363382 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194004230 con 0x7fb1a40c70c0 2024-05-08T23:32:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7fb1a8bde700 1 -- 172.21.15.33:0/2327363382 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194004390 con 0x7fb1a40c70c0 2024-05-08T23:32:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7fb1a8bde700 1 -- 172.21.15.33:0/2327363382 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940036a0 con 0x7fb1a40c70c0 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7f835a519700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83540c70c0 0x7f83540c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7f835a519700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83541577a0 0x7f8354156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7f835a519700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83541572f0 con 0x7f83540c8e50 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7f835a519700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8354157c30 con 0x7f83540c70c0 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.662+0000 7f835a519700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83540c8110 con 0x7f83541577a0 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f8353fff700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f83540c8e50 0x7f8354150480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59062/0 (socket says 172.21.15.33:59062) 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f8353fff700 1 -- 172.21.15.33:0/3755656936 learned_addr learned my addr 172.21.15.33:0/3755656936 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f8358ab6700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83541577a0 0x7f8354156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f83537fe700 1 --2- 172.21.15.33:0/3755656936 >> v2:172.21.15.42:3300/0 conn(0x7f83540c70c0 0x7f83540c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f83537fe700 1 -- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83541577a0 msgr2=0x7f8354156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f83537fe700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83541577a0 0x7f8354156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f83537fe700 1 -- 172.21.15.33:0/3755656936 >> v1:172.21.15.42:6789/0 conn(0x7f83540c8e50 legacy=0x7f8354150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f83537fe700 1 -- 172.21.15.33:0/3755656936 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540c7e30 con 0x7f83540c70c0 2024-05-08T23:32:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.663+0000 7f9138567700 1 Processor -- start 2024-05-08T23:32:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7f83537fe700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83540c7780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f834c009d90 tx=0x7f834c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc187620e387d6ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7f8352ffd700 1 -- 172.21.15.33:0/3755656936 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00e070 con 0x7f83540c70c0 2024-05-08T23:32:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7f8352ffd700 1 -- 172.21.15.33:0/3755656936 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c00bb20 con 0x7f83540c70c0 2024-05-08T23:32:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7f8352ffd700 1 -- 172.21.15.33:0/3755656936 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c012430 con 0x7f83540c70c0 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 -- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c70c0 msgr2=0x7fb1a40c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 --2- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c70c0 0x7fb1a40c7780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb194009840 tx=0x7fb194009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 -- 172.21.15.33:0/2327363382 shutdown_connections 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 --2- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 --2- 172.21.15.33:0/2327363382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c70c0 0x7fb1a40c7780 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 -- 172.21.15.33:0/2327363382 >> 172.21.15.33:0/2327363382 conn(0x7fb1a40c8360 msgr2=0x7fb1a40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7f9138567700 1 -- start start 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f835a519700 1 -- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 msgr2=0x7f83540c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f835a519700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83540c7780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f834c009d90 tx=0x7f834c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 -- 172.21.15.33:0/2327363382 shutdown_connections 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f9138567700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91300c9050 0x7f913006b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f9138567700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91300c7ae0 0x7f9130070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f9138567700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91300c5dc0 con 0x7f91301556e0 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f9138567700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9130162c90 con 0x7f91300c7ae0 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.665+0000 7f9138567700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913006b180 con 0x7f91300c9050 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.664+0000 7fb1abe44700 1 -- 172.21.15.33:0/2327363382 wait complete. 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f9135b02700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91300c9050 0x7f913006b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f9135b02700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91300c9050 0x7f913006b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53818/0 (socket says 172.21.15.33:53818) 2024-05-08T23:32:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f9135b02700 1 -- 172.21.15.33:0/4112990471 learned_addr learned my addr 172.21.15.33:0/4112990471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f1b0ea9a700 1 Processor -- start 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f1b0ea9a700 1 -- start start 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0ea9a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1b081556e0 0x7f1b0806b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0ea9a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1b080c9050 0x7f1b08070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0ea9a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b080c5dc0 con 0x7f1b080c7ae0 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0ea9a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b08162c90 con 0x7f1b081556e0 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0ea9a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0806b180 con 0x7f1b080c9050 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b07fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1b080c9050 0x7f1b08070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b07fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1b080c9050 0x7f1b08070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55054/0 (socket says 172.21.15.33:55054) 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b0d037700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1b080c7ae0 0x7f1b08070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59070/0 (socket says 172.21.15.33:59070) 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7f1b07fff700 1 -- 172.21.15.33:0/1261946296 learned_addr learned my addr 172.21.15.33:0/1261946296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f1b07fff700 1 -- 172.21.15.33:0/1261946296 >> v2:172.21.15.33:3300/0 conn(0x7f1b081556e0 msgr2=0x7f1b0806b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f1b07fff700 1 --2- 172.21.15.33:0/1261946296 >> v2:172.21.15.33:3300/0 conn(0x7f1b081556e0 0x7f1b0806b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f1b07fff700 1 -- 172.21.15.33:0/1261946296 >> v1:172.21.15.42:6789/0 conn(0x7f1b080c7ae0 legacy=0x7f1b08070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f1b07fff700 1 -- 172.21.15.33:0/1261946296 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b081629b0 con 0x7f1b080c9050 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 -- 172.21.15.33:0/3755656936 shutdown_connections 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83541577a0 0x7f8354156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 --2- 172.21.15.33:0/3755656936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83540c7780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 -- 172.21.15.33:0/3755656936 >> 172.21.15.33:0/3755656936 conn(0x7f83540c8360 msgr2=0x7f83540c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 -- 172.21.15.33:0/3755656936 shutdown_connections 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7f835a519700 1 -- 172.21.15.33:0/3755656936 wait complete. 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f835a519700 1 Processor -- start 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7fb1abe44700 1 Processor -- start 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.666+0000 7fb1abe44700 1 -- start start 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a40c70c0 0x7fb1a406ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4155160 con 0x7fb1a40c70c0 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4156650 con 0x7fb1a40c8e50 2024-05-08T23:32:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.667+0000 7fb1abe44700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4154c00 con 0x7fb1a41577a0 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1aa3e1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55060/0 (socket says 172.21.15.33:55060) 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 -- 172.21.15.33:0/3408916111 learned_addr learned my addr 172.21.15.33:0/3408916111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f1b07fff700 1 --2- 172.21.15.33:0/1261946296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c9050 0x7f1b08070be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b0806f060 tx=0x7f1b0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7359bba942f9ba10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 -- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 msgr2=0x7fb1a4151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 -- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a40c70c0 msgr2=0x7fb1a406ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a40c70c0 0x7fb1a406ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7fb1a93df700 1 -- 172.21.15.33:0/3408916111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1940094a0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7f1b077fe700 1 -- 172.21.15.33:0/1261946296 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000e070 con 0x7f1b080c9050 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7f1b077fe700 1 -- 172.21.15.33:0/1261946296 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000b9c0 con 0x7f1b080c9050 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7f1b077fe700 1 -- 172.21.15.33:0/1261946296 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000bd70 con 0x7f1b080c9050 2024-05-08T23:32:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f835a519700 1 -- start start 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb1a93df700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb194000c00 tx=0x7fb194004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f9136b04700 1 --2- 172.21.15.33:0/4112990471 >> v2:172.21.15.33:3300/0 conn(0x7f91300c7ae0 0x7f9130070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83541f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 0x7f8354200be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83541577a0 0x7f83541efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541f1360 con 0x7f83541577a0 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541f1540 con 0x7f83540c70c0 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f835a519700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83541fcc60 con 0x7f83540c8e50 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f83537fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 0x7f8354200be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f8353fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83541f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f83537fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 0x7f8354200be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53836/0 (socket says 172.21.15.33:53836) 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f83537fe700 1 -- 172.21.15.33:0/339023243 learned_addr learned my addr 172.21.15.33:0/339023243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/1261946296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c9050 msgr2=0x7f1b08070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/1261946296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c9050 0x7f1b08070be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b0806f060 tx=0x7f1b0000b330 comp rx=0 tx=0).stop 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665602603 0 0) 0x7f91300c5dc0 con 0x7f91301556e0 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f911c0035b0 con 0x7f91301556e0 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 -- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 msgr2=0x7f8354200be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 0x7f8354200be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 -- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83541577a0 msgr2=0x7f83541efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83541577a0 0x7f83541efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 -- 172.21.15.33:0/339023243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009a70 con 0x7f83540c70c0 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194004580 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194011630 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940119e0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a4154ea0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4156ba0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c0052f0 con 0x7fb1a40c8e50 2024-05-08T23:32:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8353fff700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83541f0e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f834800c430 tx=0x7f834800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2051011104 0 0) 0x7f911c0035b0 con 0x7f91301556e0 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91300c9050 msgr2=0x7f913006b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91300c9050 0x7f913006b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 >> v2:172.21.15.33:3300/0 conn(0x7f91300c7ae0 msgr2=0x7f9130070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 --2- 172.21.15.33:0/4112990471 >> v2:172.21.15.33:3300/0 conn(0x7f91300c7ae0 0x7f9130070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91301629b0 con 0x7f91301556e0 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f9120001670 con 0x7f91301556e0 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348006310 con 0x7f83540c70c0 2024-05-08T23:32:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8354201120 con 0x7f83540c70c0 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 >> v1:172.21.15.33:6789/0 conn(0x7f91301556e0 legacy=0x7f9130070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c003a00 0x7f911c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f911c004850 0x7f911c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f911c005210 0x7f911c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130162c90 con 0x7f911c003a00 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913006b180 con 0x7f911c004850 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c00bf50 con 0x7f911c005210 2024-05-08T23:32:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83541fda80 con 0x7f83540c70c0 2024-05-08T23:32:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.672+0000 7f9136b04700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f911c005210 0x7f911c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8354075460 con 0x7f83540c70c0 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136303700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c003a00 0x7f911c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 -- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f911c004850 msgr2=0x7f911c004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f911c004850 0x7f911c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 -- 172.21.15.33:0/4112990471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c003a00 msgr2=0x7f911c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911c003a00 0x7f911c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 -- 172.21.15.33:0/4112990471 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91301629b0 con 0x7f911c005210 2024-05-08T23:32:19.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7f9136b04700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f911c005210 0x7f911c00ba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91300c9710 tx=0x7f911c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5979ba4927af282 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb194018050 con 0x7fb1a40c8e50 2024-05-08T23:32:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7fb19a7fc700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 0x7fb18806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.673+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb19408a110 con 0x7fb1a40c8e50 2024-05-08T23:32:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.674+0000 7fb1a9be0700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 0x7fb18806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.674+0000 7fb1a9be0700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 0x7fb18806d250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c007da0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.668+0000 7f67bbcf5700 1 Processor -- start 2024-05-08T23:32:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.669+0000 7f67bbcf5700 1 -- start start 2024-05-08T23:32:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c58c0 0x7f67b4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67b40c9050 0x7f67b4070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67b41670c0 0x7f67b4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b4072e40 con 0x7f67b40c9050 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b406b920 con 0x7f67b41670c0 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67bbcf5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b406aa30 con 0x7f67b40c58c0 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67ba292700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67b41670c0 0x7f67b4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67ba292700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f67b41670c0 0x7f67b4070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55080/0 (socket says 172.21.15.33:55080) 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.670+0000 7f67ba292700 1 -- 172.21.15.33:0/2241088324 learned_addr learned my addr 172.21.15.33:0/2241088324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f67b9a91700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c58c0 0x7f67b4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.671+0000 7f67b9290700 1 --2- 172.21.15.33:0/2241088324 >> v2:172.21.15.33:3300/0 conn(0x7f67b40c9050 0x7f67b4070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 -- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c58c0 msgr2=0x7f67b4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c58c0 0x7f67b4070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 -- 172.21.15.33:0/2241088324 >> v2:172.21.15.33:3300/0 conn(0x7f67b40c9050 msgr2=0x7f67b4070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 --2- 172.21.15.33:0/2241088324 >> v2:172.21.15.33:3300/0 conn(0x7f67b40c9050 0x7f67b4070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 -- 172.21.15.33:0/2241088324 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b406ad10 con 0x7f67b41670c0 2024-05-08T23:32:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.675+0000 7f67ba292700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41670c0 0x7f67b4070cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67b416b6d0 tx=0x7f67ac00a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ec21ae15807eb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800ebd0 con 0x7f911c005210 2024-05-08T23:32:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912800f040 con 0x7f911c005210 2024-05-08T23:32:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f9135301700 1 -- 172.21.15.33:0/4112990471 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128014480 con 0x7f911c005210 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f67b8a8f700 1 -- 172.21.15.33:0/2241088324 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd30 con 0x7f67b41670c0 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800ce90 con 0x7f83540c70c0 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348006690 con 0x7f83540c70c0 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f9138567700 1 -- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f911c005210 msgr2=0x7f911c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f9138567700 1 --2- 172.21.15.33:0/4112990471 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f911c005210 0x7f911c00ba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91300c9710 tx=0x7f911c005880 comp rx=0 tx=0).stop 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f9136b04700 1 -- 172.21.15.33:0/4112990471 reap_dead start 2024-05-08T23:32:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f9138567700 1 -- 172.21.15.33:0/4112990471 shutdown_connections 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f9138567700 1 -- 172.21.15.33:0/4112990471 >> 172.21.15.33:0/4112990471 conn(0x7f91300c8360 msgr2=0x7f91300c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f9138567700 1 -- 172.21.15.33:0/4112990471 shutdown_connections 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/1261946296 shutdown_connections 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/1261946296 >> v2:172.21.15.33:3300/0 conn(0x7f1b081556e0 0x7f1b0806b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/1261946296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c9050 0x7f1b08070be0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/1261946296 >> 172.21.15.33:0/1261946296 conn(0x7f1b080c8360 msgr2=0x7f1b080c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/1261946296 shutdown_connections 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/1261946296 wait complete. 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f1b0ea9a700 1 Processor -- start 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f1b0ea9a700 1 -- start start 2024-05-08T23:32:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 0x7f1b08161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1b080c9050 0x7f1b081fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 0x7f1b081e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b081e8360 con 0x7f1b081556e0 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b08200f20 con 0x7f1b080c7ae0 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f1b0ea9a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b081fd100 con 0x7f1b080c9050 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f1b0c836700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 0x7f1b08161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f1b0c836700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 0x7f1b08161a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55092/0 (socket says 172.21.15.33:55092) 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f1b0c836700 1 -- 172.21.15.33:0/7375018 learned_addr learned my addr 172.21.15.33:0/7375018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f1b07fff700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1b080c9050 0x7f1b081fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8348011030 con 0x7f83540c70c0 2024-05-08T23:32:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f8350ff9700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 0x7f833806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8348088e90 con 0x7f83540c70c0 2024-05-08T23:32:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f83537fe700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 0x7f833806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb194055360 con 0x7fb1a40c8e50 2024-05-08T23:32:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f9138567700 1 -- 172.21.15.33:0/4112990471 wait complete. 2024-05-08T23:32:19.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f83537fe700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 0x7f833806d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83540c4960 tx=0x7f834c011000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.681+0000 7f1b0d037700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 0x7f1b081e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f1b0d037700 1 -- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1b080c9050 msgr2=0x7f1b081fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f1b0d037700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1b080c9050 0x7f1b081fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f1b0d037700 1 -- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 msgr2=0x7f1b08161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f1b0d037700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 0x7f1b08161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.682+0000 7f1b0d037700 1 -- 172.21.15.33:0/7375018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009b30 con 0x7f1b081556e0 2024-05-08T23:32:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8348054050 con 0x7f83540c70c0 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f67b8a8f700 1 -- 172.21.15.33:0/2241088324 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00ce90 con 0x7f67b41670c0 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.677+0000 7f67b8a8f700 1 -- 172.21.15.33:0/2241088324 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac003c40 con 0x7f67b41670c0 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f67bbcf5700 1 -- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41670c0 msgr2=0x7f67b4070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.678+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41670c0 0x7f67b4070cc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f67b416b6d0 tx=0x7f67ac00a850 comp rx=0 tx=0).stop 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 -- 172.21.15.33:0/2241088324 shutdown_connections 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b41670c0 0x7f67b4070cc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/2241088324 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c58c0 0x7f67b4070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/2241088324 >> v2:172.21.15.33:3300/0 conn(0x7f67b40c9050 0x7f67b4070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 -- 172.21.15.33:0/2241088324 >> 172.21.15.33:0/2241088324 conn(0x7f67b40c8360 msgr2=0x7f67b40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.679+0000 7f67bbcf5700 1 -- 172.21.15.33:0/2241088324 shutdown_connections 2024-05-08T23:32:19.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.680+0000 7f67bbcf5700 1 -- 172.21.15.33:0/2241088324 wait complete. 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f67bbcf5700 1 Processor -- start 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f67bbcf5700 1 -- start start 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b40c58c0 0x7f67b40ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 0x7f67b40c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b416cad0 con 0x7f67b41670c0 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4160160 con 0x7f67b40c58c0 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67bbcf5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4168350 con 0x7f67b40c9050 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67b9290700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67ba292700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 0x7f67b40c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67b9290700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53852/0 (socket says 172.21.15.33:53852) 2024-05-08T23:32:19.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f67b9290700 1 -- 172.21.15.33:0/3842851528 learned_addr learned my addr 172.21.15.33:0/3842851528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f9138567700 1 Processor -- start 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f9138567700 1 -- start start 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 0x7f9130151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9130151d40 0x7f9130150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913014f510 con 0x7f9130151d40 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130150090 con 0x7f9130151420 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.684+0000 7f9138567700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91301f4cd0 con 0x7f913015f650 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:53856/0 (socket says 172.21.15.33:53856) 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 -- 172.21.15.33:0/4204327605 learned_addr learned my addr 172.21.15.33:0/4204327605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136303700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 0x7f9130151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 -- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 msgr2=0x7f9130151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 0x7f9130151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 -- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9130151d40 msgr2=0x7f9130150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9130151d40 0x7f9130150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 -- 172.21.15.33:0/4204327605 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913014f760 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136303700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 0x7f9130151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f9136b04700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f912800e700 tx=0x7f9128009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007000 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128014690 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913014fa40 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91300caee0 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.688+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91140052f0 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.688+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128014a40 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.690+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9128014ba0 con 0x7f913015f650 2024-05-08T23:32:19.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.691+0000 7f9126ffd700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 0x7f911c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.691+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f912809bb30 con 0x7f913015f650 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.683+0000 7f1b0d037700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 0x7f1b081e84e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c3c0 tx=0x7f1af800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67b9a91700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b40c58c0 0x7f67b40ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 -- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 msgr2=0x7f67b4155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 -- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b40c58c0 msgr2=0x7f67b40ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b40c58c0 0x7f67b40ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 -- 172.21.15.33:0/3842851528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac00bb70 con 0x7f67b41670c0 2024-05-08T23:32:19.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67b9290700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.685+0000 7f67ba292700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 0x7f67b40c4450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67b406cf50 tx=0x7f67ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac003e30 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b4168630 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.686+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b415d3d0 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.687+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0073e0 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.687+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b40c7f10 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.688+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac008270 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.690+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f67ac0083d0 con 0x7f67b41670c0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.693+0000 7f9136303700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 0x7f911c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.693+0000 7f9136303700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 0x7f911c06e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9120000ff0 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.694+0000 7f67aa7fc700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 0x7f679806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.694+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67ac08a200 con 0x7f67b41670c0 2024-05-08T23:32:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.697+0000 7f67b9a91700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 0x7f679806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.698+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f67ac0553d0 con 0x7f67b41670c0 2024-05-08T23:32:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.698+0000 7f67b9a91700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 0x7f679806d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67b0006280 tx=0x7f67b0010040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.703+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8005cb0 con 0x7f1b081556e0 2024-05-08T23:32:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.706+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9128066d80 con 0x7f913015f650 2024-05-08T23:32:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.703+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8005e10 con 0x7f1b081556e0 2024-05-08T23:32:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.703+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80072d0 con 0x7f1b081556e0 2024-05-08T23:32:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.703+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b08201110 con 0x7f1b081556e0 2024-05-08T23:32:19.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.703+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081fd680 con 0x7f1b081556e0 2024-05-08T23:32:19.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.706+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b080c7f90 con 0x7f1b081556e0 2024-05-08T23:32:19.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.707+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1af8007430 con 0x7f1b081556e0 2024-05-08T23:32:19.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.708+0000 7f1b057fa700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 0x7f1aec06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.714+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1af8057450 con 0x7f1b081556e0 2024-05-08T23:32:19.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.717+0000 7f1b0c836700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 0x7f1aec06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.718+0000 7f1b0c836700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 0x7f1aec06d1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0098f0 tx=0x7f1afc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.718+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1af8053790 con 0x7f1b081556e0 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.805+0000 7f09b667f700 1 Processor -- start 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.805+0000 7f09b667f700 1 -- start start 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.806+0000 7f09b667f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f09b00c7780 0x7f09b01577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.806+0000 7f09b667f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00c8110 con 0x7f09b00c70c0 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.806+0000 7f09b667f700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00c7e30 con 0x7f09b0150510 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.806+0000 7f09b667f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00c7c10 con 0x7f09b00c7780 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.807+0000 7f09af7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f09b00c70c0 0x7f09b015ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52076/0 (socket says 172.21.15.33:52076) 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.807+0000 7f09af7fe700 1 -- 172.21.15.33:0/1468401387 learned_addr learned my addr 172.21.15.33:0/1468401387 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:19.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.807+0000 7f09b4c1c700 1 --2- 172.21.15.33:0/1468401387 >> v2:172.21.15.42:3300/0 conn(0x7f09b00c7780 0x7f09b01577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.808+0000 7f09aeffd700 1 -- 172.21.15.33:0/1468401387 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259082462 0 0) 0x7f09b00c8110 con 0x7f09b00c70c0 2024-05-08T23:32:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.808+0000 7f09aeffd700 1 -- 172.21.15.33:0/1468401387 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09980035b0 con 0x7f09b00c70c0 2024-05-08T23:32:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.809+0000 7f09b4c1c700 1 -- 172.21.15.33:0/1468401387 >> v1:172.21.15.42:6789/0 conn(0x7f09b0150510 legacy=0x7f09b006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.809+0000 7f09b4c1c700 1 -- 172.21.15.33:0/1468401387 >> v1:172.21.15.33:6789/0 conn(0x7f09b00c70c0 legacy=0x7f09b015ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.809+0000 7f09b4c1c700 1 -- 172.21.15.33:0/1468401387 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00c6300 con 0x7f09b00c7780 2024-05-08T23:32:19.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.809+0000 7f09b4c1c700 1 --2- 172.21.15.33:0/1468401387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00c7780 0x7f09b01577a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f09a00091b0 tx=0x7f09a00094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b602d2038e4b4766 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.810+0000 7f09aeffd700 1 -- 172.21.15.33:0/1468401387 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0004230 con 0x7f09b00c7780 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.810+0000 7f09aeffd700 1 -- 172.21.15.33:0/1468401387 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0004390 con 0x7f09b00c7780 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.810+0000 7f09aeffd700 1 -- 172.21.15.33:0/1468401387 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000f740 con 0x7f09b00c7780 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 -- 172.21.15.33:0/1468401387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00c7780 msgr2=0x7f09b01577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 --2- 172.21.15.33:0/1468401387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00c7780 0x7f09b01577a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f09a00091b0 tx=0x7f09a00094c0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 -- 172.21.15.33:0/1468401387 shutdown_connections 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 --2- 172.21.15.33:0/1468401387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b00c7780 0x7f09b01577a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 -- 172.21.15.33:0/1468401387 >> 172.21.15.33:0/1468401387 conn(0x7f09b00c8360 msgr2=0x7f09b00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:19.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 -- 172.21.15.33:0/1468401387 shutdown_connections 2024-05-08T23:32:19.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.811+0000 7f09b667f700 1 -- 172.21.15.33:0/1468401387 wait complete. 2024-05-08T23:32:19.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.813+0000 7f09b667f700 1 Processor -- start 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.814+0000 7f09b667f700 1 -- start start 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f09b00c70c0 0x7f09b01491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b0150510 0x7f09b0149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0162150 con 0x7f09b00c7780 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0145cf0 con 0x7f09b0150510 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.815+0000 7f09b667f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0147820 con 0x7f09b00c70c0 2024-05-08T23:32:19.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53106/0 (socket says 172.21.15.33:53106) 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 -- 172.21.15.33:0/1769763040 learned_addr learned my addr 172.21.15.33:0/1769763040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 -- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f09b00c70c0 msgr2=0x7f09b01491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f09b00c70c0 0x7f09b01491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.816+0000 7f09af7fe700 1 -- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b0150510 msgr2=0x7f09b0149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.817+0000 7f09af7fe700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b0150510 0x7f09b0149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.817+0000 7f09af7fe700 1 -- 172.21.15.33:0/1769763040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009d30 con 0x7f09b00c7780 2024-05-08T23:32:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.817+0000 7f09af7fe700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cbe0 tx=0x7f09a800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.818+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800abc0 con 0x7f09b00c7780 2024-05-08T23:32:19.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.818+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800ad20 con 0x7f09b00c7780 2024-05-08T23:32:19.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.818+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8007310 con 0x7f09b00c7780 2024-05-08T23:32:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.818+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0161e80 con 0x7f09b00c7780 2024-05-08T23:32:19.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.818+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b0147a70 con 0x7f09b00c7780 2024-05-08T23:32:19.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.822+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f09a8007470 con 0x7f09b00c7780 2024-05-08T23:32:19.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.823+0000 7f09acff9700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 0x7f099806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:19.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.823+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09a80891d0 con 0x7f09b00c7780 2024-05-08T23:32:19.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.825+0000 7f09affff700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 0x7f099806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:19.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.825+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09900052f0 con 0x7f09b00c7780 2024-05-08T23:32:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.826+0000 7f09affff700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 0x7f099806d6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09a40015f0 tx=0x7f09a4007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:19.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.836+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f09a8054420 con 0x7f09b00c7780 2024-05-08T23:32:19.997 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-05-08T23:32:19.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.995+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f81a41566d0 con 0x7f81a40c6310 2024-05-08T23:32:19.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:19.996+0000 7f818bfff700 1 -- 172.21.15.33:0/604837834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8194054510 con 0x7f81a40c6310 2024-05-08T23:32:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.000+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 msgr2=0x7f819006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.000+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 0x7f819006d250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81980090b0 tx=0x7f8198012000 comp rx=0 tx=0).stop 2024-05-08T23:32:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.000+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 msgr2=0x7f81a41f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.000+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f819400b580 tx=0x7f819400b940 comp rx=0 tx=0).stop 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 shutdown_connections 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f819006ad90 0x7f819006d250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81a40c68e0 0x7f81a41f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81a4071e20 0x7f81a415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 --2- 172.21.15.33:0/604837834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a40c6310 0x7f81a41f68f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 >> 172.21.15.33:0/604837834 conn(0x7f81a406d2b0 msgr2=0x7f81a40c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 shutdown_connections 2024-05-08T23:32:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.001+0000 7f81aa244700 1 -- 172.21.15.33:0/604837834 wait complete. 2024-05-08T23:32:20.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-08T23:32:20.023 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.098+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb2f00055a0 con 0x7fb314154160 2024-05-08T23:32:20.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.099+0000 7fb3037fe700 1 -- 172.21.15.33:0/3601342446 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb308053d30 con 0x7fb314154160 2024-05-08T23:32:20.099 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 msgr2=0x7fb2f806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 0x7fb2f806d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb30c018000 tx=0x7fb30c013000 comp rx=0 tx=0).stop 2024-05-08T23:32:20.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 msgr2=0x7fb3141545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb31415a940 tx=0x7fb30800eb10 comp rx=0 tx=0).stop 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 shutdown_connections 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb2f806ad60 0x7fb2f806d220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb314154160 0x7fb3141545f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2fc0037f0 0x7fb314153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 --2- 172.21.15.33:0/3601342446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb31414e0c0 0x7fb31414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.103+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 >> 172.21.15.33:0/3601342446 conn(0x7fb3140c8360 msgr2=0x7fb31414f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.104+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 shutdown_connections 2024-05-08T23:32:20.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.104+0000 7fb3193fd700 1 -- 172.21.15.33:0/3601342446 wait complete. 2024-05-08T23:32:20.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-05-08T23:32:20.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.124+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67b4075460 con 0x7f67b41670c0 2024-05-08T23:32:20.125 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.126+0000 7f67aa7fc700 1 -- 172.21.15.33:0/3842851528 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67ac0589f0 con 0x7f67b41670c0 2024-05-08T23:32:20.126 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.128+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb18c005870 con 0x7fb1a40c8e50 2024-05-08T23:32:20.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.129+0000 7fb19a7fc700 1 -- 172.21.15.33:0/3408916111 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb194016090 con 0x7fb1a40c8e50 2024-05-08T23:32:20.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.130+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 msgr2=0x7f679806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.130+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 0x7f679806d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67b0006280 tx=0x7f67b0010040 comp rx=0 tx=0).stop 2024-05-08T23:32:20.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.130+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 msgr2=0x7f67b40c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.130+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 0x7f67b40c4450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67b406cf50 tx=0x7f67ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 shutdown_connections 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f679806ac90 0x7f679806d150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67b41670c0 0x7f67b40c4450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67b40c58c0 0x7f67b40ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 --2- 172.21.15.33:0/3842851528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67b40c9050 0x7f67b4155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 >> 172.21.15.33:0/3842851528 conn(0x7f67b40c8360 msgr2=0x7f67b40c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 shutdown_connections 2024-05-08T23:32:20.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.131+0000 7f67bbcf5700 1 -- 172.21.15.33:0/3842851528 wait complete. 2024-05-08T23:32:20.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.133+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f83541572f0 con 0x7f83540c70c0 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 msgr2=0x7fb18806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 0x7fb18806d250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb19c007fd0 tx=0x7fb19c007da0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 msgr2=0x7fb1a4155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb194000c00 tx=0x7fb194004750 comp rx=0 tx=0).stop 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 shutdown_connections 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb18806ad90 0x7fb18806d250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb1a41577a0 0x7fb1a4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1a40c70c0 0x7fb1a406ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 --2- 172.21.15.33:0/3408916111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1a40c8e50 0x7fb1a4155ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 >> 172.21.15.33:0/3408916111 conn(0x7fb1a40c8360 msgr2=0x7fb1a40caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7f8350ff9700 1 -- 172.21.15.33:0/339023243 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83541572f0 con 0x7f83540c70c0 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 shutdown_connections 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.134+0000 7fb1abe44700 1 -- 172.21.15.33:0/3408916111 wait complete. 2024-05-08T23:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 msgr2=0x7f833806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 0x7f833806d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f83540c4960 tx=0x7f834c011000 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 msgr2=0x7f83541f0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83541f0e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f834800c430 tx=0x7f834800c740 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 shutdown_connections 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833806ac90 0x7f833806d150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83541577a0 0x7f83541efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83540c70c0 0x7f83541f0e20 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f834800c430 tx=0x7f834800c740 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 --2- 172.21.15.33:0/339023243 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83540c8e50 0x7f8354200be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.137+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 >> 172.21.15.33:0/339023243 conn(0x7f83540c8360 msgr2=0x7f83540c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.138+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 shutdown_connections 2024-05-08T23:32:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.138+0000 7f835a519700 1 -- 172.21.15.33:0/339023243 wait complete. 2024-05-08T23:32:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.147+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1b080c5240 con 0x7f1b081556e0 2024-05-08T23:32:20.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.147+0000 7f1b057fa700 1 -- 172.21.15.33:0/7375018 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1af8086e60 con 0x7f1b081556e0 2024-05-08T23:32:20.148 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.150+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 msgr2=0x7f1aec06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.150+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 0x7f1aec06d1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0098f0 tx=0x7f1afc008040 comp rx=0 tx=0).stop 2024-05-08T23:32:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.150+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 msgr2=0x7f1b081e84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.150+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 0x7f1b081e84e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c3c0 tx=0x7f1af800c780 comp rx=0 tx=0).stop 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 shutdown_connections 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1aec06ace0 0x7f1aec06d1a0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0098f0 tx=0x7f1afc008040 comp rx=0 tx=0).stop 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b081556e0 0x7f1b081e84e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b080c7ae0 0x7f1b08161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 --2- 172.21.15.33:0/7375018 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1b080c9050 0x7f1b081fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.151+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 >> 172.21.15.33:0/7375018 conn(0x7f1b080c8360 msgr2=0x7f1b080c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.155+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 shutdown_connections 2024-05-08T23:32:20.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.155+0000 7f1b0ea9a700 1 -- 172.21.15.33:0/7375018 wait complete. 2024-05-08T23:32:20.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-05-08T23:32:20.159 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-05-08T23:32:20.160 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-08T23:32:20.161 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.161+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09900055a0 con 0x7f09b00c7780 2024-05-08T23:32:20.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.162+0000 7f09acff9700 1 -- 172.21.15.33:0/1769763040 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09a8018020 con 0x7f09b00c7780 2024-05-08T23:32:20.162 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.163+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9114005870 con 0x7f913015f650 2024-05-08T23:32:20.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f9126ffd700 1 -- 172.21.15.33:0/4204327605 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f912806a3a0 con 0x7f913015f650 2024-05-08T23:32:20.164 INFO:teuthology.orchestra.run.smithi033.stdout:55834574853 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 msgr2=0x7f099806d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 0x7f099806d6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09a40015f0 tx=0x7f09a4007a20 comp rx=0 tx=0).stop 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 msgr2=0x7f09b01f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cbe0 tx=0x7f09a800a060 comp rx=0 tx=0).stop 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 shutdown_connections 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f099806b210 0x7f099806d6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09b0150510 0x7f09b0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f09b00c70c0 0x7f09b01491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 --2- 172.21.15.33:0/1769763040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b00c7780 0x7f09b01f5970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 >> 172.21.15.33:0/1769763040 conn(0x7f09b00c8360 msgr2=0x7f09b006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 shutdown_connections 2024-05-08T23:32:20.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.164+0000 7f09b667f700 1 -- 172.21.15.33:0/1769763040 wait complete. 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.166+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 msgr2=0x7f911c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.166+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 0x7f911c06e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9120000ff0 tx=0x7f9120009040 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.166+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 msgr2=0x7f9130150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.166+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f912800e700 tx=0x7f9128009750 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 shutdown_connections 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f911c06c2b0 0x7f911c06e770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f913015f650 0x7f9130150a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9130151420 0x7f9130151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 --2- 172.21.15.33:0/4204327605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9130151d40 0x7f9130150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 >> 172.21.15.33:0/4204327605 conn(0x7f91300c8360 msgr2=0x7f91300c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 shutdown_connections 2024-05-08T23:32:20.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.167+0000 7f9138567700 1 -- 172.21.15.33:0/4204327605 wait complete. 2024-05-08T23:32:20.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-05-08T23:32:20.172 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-05-08T23:32:20.181 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-05-08T23:32:20.182 DEBUG:teuthology.parallel:result is None 2024-05-08T23:32:20.182 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:32:20.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.361+0000 7f980211d700 1 Processor -- start 2024-05-08T23:32:20.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.361+0000 7f980211d700 1 -- start start 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f980211d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97fc154280 0x7f97fc0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f980211d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0c6ed0 con 0x7f97fc154660 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f980211d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc06b6e0 con 0x7f97fc0c8480 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f980211d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc0c7a10 con 0x7f97fc154280 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f97fb7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f97fc154280 0x7f97fc0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f97f2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97fc154660 0x7f97fc15ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52086/0 (socket says 172.21.15.33:52086) 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.362+0000 7f97f2ffd700 1 -- 172.21.15.33:0/4076448550 learned_addr learned my addr 172.21.15.33:0/4076448550 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97fb7fe700 1 -- 172.21.15.33:0/4076448550 >> v1:172.21.15.42:6789/0 conn(0x7f97fc0c8480 legacy=0x7f97fc15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97fb7fe700 1 -- 172.21.15.33:0/4076448550 >> v1:172.21.15.33:6789/0 conn(0x7f97fc154660 legacy=0x7f97fc15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97fb7fe700 1 -- 172.21.15.33:0/4076448550 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc06ab10 con 0x7f97fc154280 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97faffd700 1 -- 172.21.15.33:0/4076448550 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722050707 0 0) 0x7f97e4001110 con 0x7f97fc154660 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97fb7fe700 1 --2- 172.21.15.33:0/4076448550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc0c7380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009840 tx=0x7f97ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12f1796b0b18787 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97faffd700 1 -- 172.21.15.33:0/4076448550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec011830 con 0x7f97fc154280 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97faffd700 1 -- 172.21.15.33:0/4076448550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec011990 con 0x7f97fc154280 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.363+0000 7f97faffd700 1 -- 172.21.15.33:0/4076448550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec011d20 con 0x7f97fc154280 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 -- 172.21.15.33:0/4076448550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 msgr2=0x7f97fc0c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 --2- 172.21.15.33:0/4076448550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc0c7380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009840 tx=0x7f97ec009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 -- 172.21.15.33:0/4076448550 shutdown_connections 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 --2- 172.21.15.33:0/4076448550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc0c7380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 -- 172.21.15.33:0/4076448550 >> 172.21.15.33:0/4076448550 conn(0x7f97fc0c9480 msgr2=0x7f97fc0c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 -- 172.21.15.33:0/4076448550 shutdown_connections 2024-05-08T23:32:20.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.364+0000 7f980211d700 1 -- 172.21.15.33:0/4076448550 wait complete. 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.365+0000 7f980211d700 1 Processor -- start 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.365+0000 7f980211d700 1 -- start start 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97fc0c8480 0x7f97fc14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc154660 0x7f97fc15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc155040 con 0x7f97fc154280 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc15c030 con 0x7f97fc154660 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f980211d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0c60c0 con 0x7f97fc0c8480 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53120/0 (socket says 172.21.15.33:53120) 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 -- 172.21.15.33:0/3699185480 learned_addr learned my addr 172.21.15.33:0/3699185480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:20.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97fbfff700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc154660 0x7f97fc15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97fb7fe700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97fc0c8480 0x7f97fc14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 -- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97fc0c8480 msgr2=0x7f97fc14d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97fc0c8480 0x7f97fc14d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 -- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc154660 msgr2=0x7f97fc15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc154660 0x7f97fc15c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.366+0000 7f97f2ffd700 1 -- 172.21.15.33:0/3699185480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0094a0 con 0x7f97fc154280 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f97f2ffd700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c550 tx=0x7f97e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4004c80 con 0x7f97fc154280 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4004de0 con 0x7f97fc154280 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc1f0cc0 con 0x7f97fc154280 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007310 con 0x7f97fc154280 2024-05-08T23:32:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.367+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc1f09e0 con 0x7f97fc154280 2024-05-08T23:32:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.368+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc15c170 con 0x7f97fc154280 2024-05-08T23:32:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.369+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f97e4007470 con 0x7f97fc154280 2024-05-08T23:32:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.369+0000 7f97f8ff9700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 0x7f97dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.369+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97e4088f60 con 0x7f97fc154280 2024-05-08T23:32:20.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.369+0000 7f97fb7fe700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 0x7f97dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.369+0000 7f97fb7fe700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 0x7f97dc06d150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009b90 tx=0x7f97ec01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.374+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f97fc15c170 con 0x7f97fc154280 2024-05-08T23:32:20.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.585+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97fc06b370 con 0x7f97dc06ac90 2024-05-08T23:32:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.593+0000 7f97f8ff9700 1 -- 172.21.15.33:0/3699185480 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264689 (secure 0 0 0) 0x7f97ec011b00 con 0x7f97dc06ac90 2024-05-08T23:32:20.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 msgr2=0x7f97dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 0x7f97dc06d150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009b90 tx=0x7f97ec01c040 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 msgr2=0x7f97fc15c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c550 tx=0x7f97e400c910 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 shutdown_connections 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f97dc06ac90 0x7f97dc06d150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc154660 0x7f97fc15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f97fc0c8480 0x7f97fc14d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 --2- 172.21.15.33:0/3699185480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97fc154280 0x7f97fc15c450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 >> 172.21.15.33:0/3699185480 conn(0x7f97fc0c9480 msgr2=0x7f97fc072730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 shutdown_connections 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.596+0000 7f980211d700 1 -- 172.21.15.33:0/3699185480 wait complete. 2024-05-08T23:32:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:20.610 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-05-08T23:32:18.772391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219048,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.439241"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.345654+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:15.345654+0000","last_peered":"2024-05-08T23:32:15.345654+0000","last_clean":"2024-05-08T23:32:15.345654+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:15.345654+0000","last_undegraded":"2024-05-08T23:32:15.345654+0000","last_fullsized":"2024-05-08T23:32:15.345654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.835040+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:13.835040+0000","last_peered":"2024-05-08T23:32:13.835040+0000","last_clean":"2024-05-08T23:32:13.835040+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:13.835040+0000","last_undegraded":"2024-05-08T23:32:13.835040+0000","last_fullsized":"2024-05-08T23:32:13.835040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343181+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:15.343181+0000","last_peered":"2024-05-08T23:32:15.343181+0000","last_clean":"2024-05-08T23:32:15.343181+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:15.343181+0000","last_undegraded":"2024-05-08T23:32:15.343181+0000","last_fullsized":"2024-05-08T23:32:15.343181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343523+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:15.343523+0000","last_peered":"2024-05-08T23:32:15.343523+0000","last_clean":"2024-05-08T23:32:15.343523+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:15.343523+0000","last_undegraded":"2024-05-08T23:32:15.343523+0000","last_fullsized":"2024-05-08T23:32:15.343523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.834996+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:13.834996+0000","last_peered":"2024-05-08T23:32:13.834996+0000","last_clean":"2024-05-08T23:32:13.834996+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:13.834996+0000","last_undegraded":"2024-05-08T23:32:13.834996+0000","last_fullsized":"2024-05-08T23:32:13.834996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839936+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:13.839936+0000","last_peered":"2024-05-08T23:32:13.839936+0000","last_clean":"2024-05-08T23:32:13.839936+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:13.839936+0000","last_undegraded":"2024-05-08T23:32:13.839936+0000","last_fullsized":"2024-05-08T23:32:13.839936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342601+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:15.342601+0000","last_peered":"2024-05-08T23:32:15.342601+0000","last_clean":"2024-05-08T23:32:15.342601+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:15.342601+0000","last_undegraded":"2024-05-08T23:32:15.342601+0000","last_fullsized":"2024-05-08T23:32:15.342601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.344741+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:15.344741+0000","last_peered":"2024-05-08T23:32:15.344741+0000","last_clean":"2024-05-08T23:32:15.344741+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:15.344741+0000","last_undegraded":"2024-05-08T23:32:15.344741+0000","last_fullsized":"2024-05-08T23:32:15.344741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342373+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:15.342373+0000","last_peered":"2024-05-08T23:32:15.342373+0000","last_clean":"2024-05-08T23:32:15.342373+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:15.342373+0000","last_undegraded":"2024-05-08T23:32:15.342373+0000","last_fullsized":"2024-05-08T23:32:15.342373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.350409+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:15.350409+0000","last_peered":"2024-05-08T23:32:15.350409+0000","last_clean":"2024-05-08T23:32:15.350409+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:15.350409+0000","last_undegraded":"2024-05-08T23:32:15.350409+0000","last_fullsized":"2024-05-08T23:32:15.350409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.841119+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:13.841119+0000","last_peered":"2024-05-08T23:32:13.841119+0000","last_clean":"2024-05-08T23:32:13.841119+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:13.841119+0000","last_undegraded":"2024-05-08T23:32:13.841119+0000","last_fullsized":"2024-05-08T23:32:13.841119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340980+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:11.340980+0000","last_peered":"2024-05-08T23:32:11.340980+0000","last_clean":"2024-05-08T23:32:11.340980+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:11.340980+0000","last_undegraded":"2024-05-08T23:32:11.340980+0000","last_fullsized":"2024-05-08T23:32:11.340980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342603+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:15.342603+0000","last_peered":"2024-05-08T23:32:15.342603+0000","last_clean":"2024-05-08T23:32:15.342603+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:15.342603+0000","last_undegraded":"2024-05-08T23:32:15.342603+0000","last_fullsized":"2024-05-08T23:32:15.342603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342739+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:15.342739+0000","last_peered":"2024-05-08T23:32:15.342739+0000","last_clean":"2024-05-08T23:32:15.342739+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:15.342739+0000","last_undegraded":"2024-05-08T23:32:15.342739+0000","last_fullsized":"2024-05-08T23:32:15.342739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839350+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:13.839350+0000","last_peered":"2024-05-08T23:32:13.839350+0000","last_clean":"2024-05-08T23:32:13.839350+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:13.839350+0000","last_undegraded":"2024-05-08T23:32:13.839350+0000","last_fullsized":"2024-05-08T23:32:13.839350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343951+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:15.343951+0000","last_peered":"2024-05-08T23:32:15.343951+0000","last_clean":"2024-05-08T23:32:15.343951+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:15.343951+0000","last_undegraded":"2024-05-08T23:32:15.343951+0000","last_fullsized":"2024-05-08T23:32:15.343951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:20.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:20.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.788+0000 7fcfe83cc700 1 Processor -- start 2024-05-08T23:32:20.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.788+0000 7fcfe83cc700 1 -- start start 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.789+0000 7fcfe83cc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfe00c6310 0x7fcfe006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.789+0000 7fcfe83cc700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe01582e0 con 0x7fcfe00c68e0 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.789+0000 7fcfe83cc700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe0157e50 con 0x7fcfe0071e20 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.789+0000 7fcfe83cc700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe0157310 con 0x7fcfe00c6310 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe6168700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfe00c6310 0x7fcfe006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5967700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcfe00c68e0 0x7fcfe0071710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52094/0 (socket says 172.21.15.33:52094) 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5967700 1 -- 172.21.15.33:0/264814806 learned_addr learned my addr 172.21.15.33:0/264814806 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe6168700 1 -- 172.21.15.33:0/264814806 >> v1:172.21.15.42:6789/0 conn(0x7fcfe0071e20 legacy=0x7fcfe0161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe6168700 1 -- 172.21.15.33:0/264814806 >> v1:172.21.15.33:6789/0 conn(0x7fcfe00c68e0 legacy=0x7fcfe0071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5166700 1 -- 172.21.15.33:0/264814806 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665743655 0 0) 0x7fcfe01582e0 con 0x7fcfe00c68e0 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe6168700 1 -- 172.21.15.33:0/264814806 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00c66f0 con 0x7fcfe00c6310 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe6168700 1 --2- 172.21.15.33:0/264814806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006baa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009840 tx=0x7fcfdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b1c27cbdeb45af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5166700 1 -- 172.21.15.33:0/264814806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc004230 con 0x7fcfe00c6310 2024-05-08T23:32:20.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5166700 1 -- 172.21.15.33:0/264814806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc004390 con 0x7fcfe00c6310 2024-05-08T23:32:20.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.790+0000 7fcfe5166700 1 -- 172.21.15.33:0/264814806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0036a0 con 0x7fcfe00c6310 2024-05-08T23:32:20.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 -- 172.21.15.33:0/264814806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 msgr2=0x7fcfe006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/264814806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006baa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009840 tx=0x7fcfdc009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 -- 172.21.15.33:0/264814806 shutdown_connections 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/264814806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006baa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 -- 172.21.15.33:0/264814806 >> 172.21.15.33:0/264814806 conn(0x7fcfe006d2b0 msgr2=0x7fcfe006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 -- 172.21.15.33:0/264814806 shutdown_connections 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.791+0000 7fcfe83cc700 1 -- 172.21.15.33:0/264814806 wait complete. 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.793+0000 7fcfe83cc700 1 Processor -- start 2024-05-08T23:32:20.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.793+0000 7fcfe83cc700 1 -- start start 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcfe0071e20 0x7fcfe01e0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe00c68e0 0x7fcfe006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01e1dc0 con 0x7fcfe00c6310 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01ebb90 con 0x7fcfe00c68e0 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe83cc700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00c0400 con 0x7fcfe0071e20 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53126/0 (socket says 172.21.15.33:53126) 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 -- 172.21.15.33:0/4266569050 learned_addr learned my addr 172.21.15.33:0/4266569050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe6969700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe00c68e0 0x7fcfe006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe6168700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcfe0071e20 0x7fcfe01e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 -- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcfe0071e20 msgr2=0x7fcfe01e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcfe0071e20 0x7fcfe01e0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 -- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe00c68e0 msgr2=0x7fcfe006bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe00c68e0 0x7fcfe006bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 -- 172.21.15.33:0/4266569050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0094a0 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.794+0000 7fcfe5967700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00011e0 tx=0x7fcfd000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.795+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0004c80 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.795+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0004e00 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.795+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe00c0630 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.795+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0007310 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.795+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe01ec080 con 0x7fcfe00c6310 2024-05-08T23:32:20.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.796+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0075460 con 0x7fcfe00c6310 2024-05-08T23:32:20.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.796+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcfd0007470 con 0x7fcfe00c6310 2024-05-08T23:32:20.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.797+0000 7fcfd6ffd700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 0x7fcfc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:20.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.797+0000 7fcfe6168700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 0x7fcfc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:20.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.797+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfd0088f20 con 0x7fcfe00c6310 2024-05-08T23:32:20.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.797+0000 7fcfe6168700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 0x7fcfc406d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009b90 tx=0x7fcfdc01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:20.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:20.802+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcfd0054190 con 0x7fcfe00c6310 2024-05-08T23:32:21.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.016+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfe0157800 con 0x7fcfc406ac90 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.024+0000 7fcfd6ffd700 1 -- 172.21.15.33:0/4266569050 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264551 (secure 0 0 0) 0x7fcfdc004130 con 0x7fcfc406ac90 2024-05-08T23:32:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 msgr2=0x7fcfc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 0x7fcfc406d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009b90 tx=0x7fcfdc01c040 comp rx=0 tx=0).stop 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 msgr2=0x7fcfe006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00011e0 tx=0x7fcfd000ca30 comp rx=0 tx=0).stop 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 shutdown_connections 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcfc406ac90 0x7fcfc406d150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe00c68e0 0x7fcfe006bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcfe0071e20 0x7fcfe01e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 --2- 172.21.15.33:0/4266569050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfe00c6310 0x7fcfe006b9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 >> 172.21.15.33:0/4266569050 conn(0x7fcfe006d2b0 msgr2=0x7fcfe00cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 shutdown_connections 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.028+0000 7fcfe83cc700 1 -- 172.21.15.33:0/4266569050 wait complete. 2024-05-08T23:32:21.031 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:21.042 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-08T23:32:20.773050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219048,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002250"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.345654+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:15.345654+0000","last_peered":"2024-05-08T23:32:15.345654+0000","last_clean":"2024-05-08T23:32:15.345654+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:15.345654+0000","last_undegraded":"2024-05-08T23:32:15.345654+0000","last_fullsized":"2024-05-08T23:32:15.345654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.835040+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:13.835040+0000","last_peered":"2024-05-08T23:32:13.835040+0000","last_clean":"2024-05-08T23:32:13.835040+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:13.835040+0000","last_undegraded":"2024-05-08T23:32:13.835040+0000","last_fullsized":"2024-05-08T23:32:13.835040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343181+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:15.343181+0000","last_peered":"2024-05-08T23:32:15.343181+0000","last_clean":"2024-05-08T23:32:15.343181+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:15.343181+0000","last_undegraded":"2024-05-08T23:32:15.343181+0000","last_fullsized":"2024-05-08T23:32:15.343181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343523+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:15.343523+0000","last_peered":"2024-05-08T23:32:15.343523+0000","last_clean":"2024-05-08T23:32:15.343523+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:15.343523+0000","last_undegraded":"2024-05-08T23:32:15.343523+0000","last_fullsized":"2024-05-08T23:32:15.343523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.834996+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:13.834996+0000","last_peered":"2024-05-08T23:32:13.834996+0000","last_clean":"2024-05-08T23:32:13.834996+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:13.834996+0000","last_undegraded":"2024-05-08T23:32:13.834996+0000","last_fullsized":"2024-05-08T23:32:13.834996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839936+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:13.839936+0000","last_peered":"2024-05-08T23:32:13.839936+0000","last_clean":"2024-05-08T23:32:13.839936+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:13.839936+0000","last_undegraded":"2024-05-08T23:32:13.839936+0000","last_fullsized":"2024-05-08T23:32:13.839936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342601+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:15.342601+0000","last_peered":"2024-05-08T23:32:15.342601+0000","last_clean":"2024-05-08T23:32:15.342601+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:15.342601+0000","last_undegraded":"2024-05-08T23:32:15.342601+0000","last_fullsized":"2024-05-08T23:32:15.342601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.344741+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:15.344741+0000","last_peered":"2024-05-08T23:32:15.344741+0000","last_clean":"2024-05-08T23:32:15.344741+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:15.344741+0000","last_undegraded":"2024-05-08T23:32:15.344741+0000","last_fullsized":"2024-05-08T23:32:15.344741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342373+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:15.342373+0000","last_peered":"2024-05-08T23:32:15.342373+0000","last_clean":"2024-05-08T23:32:15.342373+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:15.342373+0000","last_undegraded":"2024-05-08T23:32:15.342373+0000","last_fullsized":"2024-05-08T23:32:15.342373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.350409+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:15.350409+0000","last_peered":"2024-05-08T23:32:15.350409+0000","last_clean":"2024-05-08T23:32:15.350409+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:15.350409+0000","last_undegraded":"2024-05-08T23:32:15.350409+0000","last_fullsized":"2024-05-08T23:32:15.350409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.841119+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:13.841119+0000","last_peered":"2024-05-08T23:32:13.841119+0000","last_clean":"2024-05-08T23:32:13.841119+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:13.841119+0000","last_undegraded":"2024-05-08T23:32:13.841119+0000","last_fullsized":"2024-05-08T23:32:13.841119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340980+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:11.340980+0000","last_peered":"2024-05-08T23:32:11.340980+0000","last_clean":"2024-05-08T23:32:11.340980+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:11.340980+0000","last_undegraded":"2024-05-08T23:32:11.340980+0000","last_fullsized":"2024-05-08T23:32:11.340980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342603+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:15.342603+0000","last_peered":"2024-05-08T23:32:15.342603+0000","last_clean":"2024-05-08T23:32:15.342603+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:15.342603+0000","last_undegraded":"2024-05-08T23:32:15.342603+0000","last_fullsized":"2024-05-08T23:32:15.342603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342739+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:15.342739+0000","last_peered":"2024-05-08T23:32:15.342739+0000","last_clean":"2024-05-08T23:32:15.342739+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:15.342739+0000","last_undegraded":"2024-05-08T23:32:15.342739+0000","last_fullsized":"2024-05-08T23:32:15.342739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839350+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:13.839350+0000","last_peered":"2024-05-08T23:32:13.839350+0000","last_clean":"2024-05-08T23:32:13.839350+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:13.839350+0000","last_undegraded":"2024-05-08T23:32:13.839350+0000","last_fullsized":"2024-05-08T23:32:13.839350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343951+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:15.343951+0000","last_peered":"2024-05-08T23:32:15.343951+0000","last_clean":"2024-05-08T23:32:15.343951+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:15.343951+0000","last_undegraded":"2024-05-08T23:32:15.343951+0000","last_fullsized":"2024-05-08T23:32:15.343951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0019999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6819999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1709999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.109999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0389999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1230000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:21.046 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:32:21.046 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:32:21.046 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:32:21.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T23:32:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.220+0000 7fbe43b8c700 1 Processor -- start 2024-05-08T23:32:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.220+0000 7fbe43b8c700 1 -- start start 2024-05-08T23:32:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe43b8c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 0x7fbe3c0c3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe43b8c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c06ce60 con 0x7fbe3c072b10 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe43b8c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c15b470 con 0x7fbe3c1511c0 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe43b8c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c90a0 con 0x7fbe3c15af30 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe41127700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbe3c072b10 0x7fbe3c0c2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52104/0 (socket says 172.21.15.33:52104) 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe41127700 1 -- 172.21.15.33:0/349580608 learned_addr learned my addr 172.21.15.33:0/349580608 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe42129700 1 --2- 172.21.15.33:0/349580608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 0x7fbe3c0c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35455909 0 0) 0x7fbe3c06ce60 con 0x7fbe3c072b10 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.221+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe280035b0 con 0x7fbe3c072b10 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625683329 0 0) 0x7fbe3c15b470 con 0x7fbe3c1511c0 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3c06ce60 con 0x7fbe3c1511c0 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe42129700 1 -- 172.21.15.33:0/349580608 >> v1:172.21.15.42:6789/0 conn(0x7fbe3c1511c0 legacy=0x7fbe3c072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe42129700 1 -- 172.21.15.33:0/349580608 >> v1:172.21.15.33:6789/0 conn(0x7fbe3c072b10 legacy=0x7fbe3c0c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe42129700 1 -- 172.21.15.33:0/349580608 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0c9380 con 0x7fbe3c15af30 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe42129700 1 --2- 172.21.15.33:0/349580608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 0x7fbe3c0c3610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009dc0 tx=0x7fbe3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30341026938dda1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400e070 con 0x7fbe3c15af30 2024-05-08T23:32:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3400bb20 con 0x7fbe3c15af30 2024-05-08T23:32:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.222+0000 7fbe40926700 1 -- 172.21.15.33:0/349580608 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400be90 con 0x7fbe3c15af30 2024-05-08T23:32:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 -- 172.21.15.33:0/349580608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 msgr2=0x7fbe3c0c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/349580608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 0x7fbe3c0c3610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009dc0 tx=0x7fbe3400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 -- 172.21.15.33:0/349580608 shutdown_connections 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/349580608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c15af30 0x7fbe3c0c3610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 -- 172.21.15.33:0/349580608 >> 172.21.15.33:0/349580608 conn(0x7fbe3c06e990 msgr2=0x7fbe3c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 -- 172.21.15.33:0/349580608 shutdown_connections 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.223+0000 7fbe43b8c700 1 -- 172.21.15.33:0/349580608 wait complete. 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.224+0000 7fbe43b8c700 1 Processor -- start 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.224+0000 7fbe43b8c700 1 -- start start 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c1511c0 0x7fbe3c1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe3c15af30 0x7fbe3c0cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c146800 con 0x7fbe3c072b10 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cc890 con 0x7fbe3c15af30 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe43b8c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c142760 con 0x7fbe3c1511c0 2024-05-08T23:32:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53128/0 (socket says 172.21.15.33:53128) 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 -- 172.21.15.33:0/3675401063 learned_addr learned my addr 172.21.15.33:0/3675401063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41127700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c1511c0 0x7fbe3c1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe42129700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe3c15af30 0x7fbe3c0cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 -- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c1511c0 msgr2=0x7fbe3c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c1511c0 0x7fbe3c1477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 -- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe3c15af30 msgr2=0x7fbe3c0cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe3c15af30 0x7fbe3c0cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.225+0000 7fbe41928700 1 -- 172.21.15.33:0/3675401063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34009a70 con 0x7fbe3c072b10 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe41928700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c520 tx=0x7fbe3800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380052d0 con 0x7fbe3c072b10 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38005430 con 0x7fbe3c072b10 2024-05-08T23:32:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c143250 con 0x7fbe3c072b10 2024-05-08T23:32:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380038c0 con 0x7fbe3c072b10 2024-05-08T23:32:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.226+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c142c90 con 0x7fbe3c072b10 2024-05-08T23:32:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.227+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c075460 con 0x7fbe3c072b10 2024-05-08T23:32:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.227+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbe38003a20 con 0x7fbe3c072b10 2024-05-08T23:32:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.228+0000 7fbe327fc700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.228+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe380887c0 con 0x7fbe3c072b10 2024-05-08T23:32:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.228+0000 7fbe41127700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.228+0000 7fbe41127700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 0x7fbe2006d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c149a00 tx=0x7fbe2c007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.232+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbe38053910 con 0x7fbe3c072b10 2024-05-08T23:32:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.502+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbe3c0c77e0 con 0x7fbe3c072b10 2024-05-08T23:32:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.502+0000 7fbe327fc700 1 -- 172.21.15.33:0/3675401063 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbe38017040 con 0x7fbe3c072b10 2024-05-08T23:32:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:21.502 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.504+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 msgr2=0x7fbe2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.504+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 0x7fbe2006d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c149a00 tx=0x7fbe2c007dc0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 msgr2=0x7fbe3c147f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c520 tx=0x7fbe3800c830 comp rx=0 tx=0).stop 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 shutdown_connections 2024-05-08T23:32:21.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbe2006ac90 0x7fbe2006d150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe3c15af30 0x7fbe3c0cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe3c072b10 0x7fbe3c147f60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 --2- 172.21.15.33:0/3675401063 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbe3c1511c0 0x7fbe3c1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 >> 172.21.15.33:0/3675401063 conn(0x7fbe3c06e990 msgr2=0x7fbe3c14f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 shutdown_connections 2024-05-08T23:32:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.505+0000 7fbe43b8c700 1 -- 172.21.15.33:0/3675401063 wait complete. 2024-05-08T23:32:21.519 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:32:21.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.654+0000 7f90615db700 1 Processor -- start 2024-05-08T23:32:21.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.654+0000 7f90615db700 1 -- start start 2024-05-08T23:32:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f90615db700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f905c0ce030 0x7f905c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f90615db700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f905c0ce410 0x7f905c0c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f90615db700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c15a930 con 0x7f905c070990 2024-05-08T23:32:21.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f90615db700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c15ac10 con 0x7f905c0ce030 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f90615db700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c15aef0 con 0x7f905c0ce410 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f905c0ce030 0x7f905c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f905c0ce030 0x7f905c15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53130/0 (socket says 172.21.15.33:53130) 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 -- 172.21.15.33:0/1305348791 learned_addr learned my addr 172.21.15.33:0/1305348791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905b7fe700 1 --2- 172.21.15.33:0/1305348791 >> v2:172.21.15.42:3300/0 conn(0x7f905c0ce410 0x7f905c0c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 -- 172.21.15.33:0/1305348791 >> v2:172.21.15.42:3300/0 conn(0x7f905c0ce410 msgr2=0x7f905c0c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 --2- 172.21.15.33:0/1305348791 >> v2:172.21.15.42:3300/0 conn(0x7f905c0ce410 0x7f905c0c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 -- 172.21.15.33:0/1305348791 >> v1:172.21.15.42:6789/0 conn(0x7f905c070990 legacy=0x7f905c1541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 -- 172.21.15.33:0/1305348791 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0c8fb0 con 0x7f905c0ce030 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f905affd700 1 --2- 172.21.15.33:0/1305348791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce030 0x7f905c15a2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f905c0b41e0 tx=0x7f904c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d48d6febdd108 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.655+0000 7f9059ffb700 1 -- 172.21.15.33:0/1305348791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00b870 con 0x7f905c0ce030 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f9059ffb700 1 -- 172.21.15.33:0/1305348791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00b9d0 con 0x7f905c0ce030 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f9059ffb700 1 -- 172.21.15.33:0/1305348791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00bd60 con 0x7f905c0ce030 2024-05-08T23:32:21.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 -- 172.21.15.33:0/1305348791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce030 msgr2=0x7f905c15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 --2- 172.21.15.33:0/1305348791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce030 0x7f905c15a2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f905c0b41e0 tx=0x7f904c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 -- 172.21.15.33:0/1305348791 shutdown_connections 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 --2- 172.21.15.33:0/1305348791 >> v2:172.21.15.42:3300/0 conn(0x7f905c0ce410 0x7f905c0c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 --2- 172.21.15.33:0/1305348791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce030 0x7f905c15a2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 -- 172.21.15.33:0/1305348791 >> 172.21.15.33:0/1305348791 conn(0x7f905c072230 msgr2=0x7f905c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 -- 172.21.15.33:0/1305348791 shutdown_connections 2024-05-08T23:32:21.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.656+0000 7f90615db700 1 -- 172.21.15.33:0/1305348791 wait complete. 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.658+0000 7f90615db700 1 Processor -- start 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.658+0000 7f90615db700 1 -- start start 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f905c070990 0x7f905c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 0x7f905c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c158160 con 0x7f905c0ce410 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c0c6400 con 0x7f905c070990 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f90615db700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c156f20 con 0x7f905c0ce030 2024-05-08T23:32:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f905b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f905b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53146/0 (socket says 172.21.15.33:53146) 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f905b7fe700 1 -- 172.21.15.33:0/2243791361 learned_addr learned my addr 172.21.15.33:0/2243791361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f905affd700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f905c070990 0x7f905c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.659+0000 7f905a7fc700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 0x7f905c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 -- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 msgr2=0x7f905c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 0x7f905c14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 -- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f905c070990 msgr2=0x7f905c14dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f905c070990 0x7f905c14dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 -- 172.21.15.33:0/2243791361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009550 con 0x7f905c0ce410 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905a7fc700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 0x7f905c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f905b7fe700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9054007d80 tx=0x7f905400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905400ede0 con 0x7f905c0ce410 2024-05-08T23:32:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9054014420 con 0x7f905c0ce410 2024-05-08T23:32:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c1571c0 con 0x7f905c0ce410 2024-05-08T23:32:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540147d0 con 0x7f905c0ce410 2024-05-08T23:32:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.660+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c157720 con 0x7f905c0ce410 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.661+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c06d220 con 0x7f905c0ce410 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.661+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9054014930 con 0x7f905c0ce410 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.662+0000 7f90537fe700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 0x7f903c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.662+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90540899d0 con 0x7f905c0ce410 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.662+0000 7f905affd700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 0x7f903c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.662+0000 7f905affd700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 0x7f903c06d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.667+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9054054bb0 con 0x7f905c0ce410 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.885+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905c075460 con 0x7f905c0ce410 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.886+0000 7f90537fe700 1 -- 172.21.15.33:0/2243791361 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f90540581d0 con 0x7f905c0ce410 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 msgr2=0x7f903c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 0x7f903c06d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c008030 comp rx=0 tx=0).stop 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 msgr2=0x7f905c14d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9054007d80 tx=0x7f905400e3c0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 shutdown_connections 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f903c06ac90 0x7f903c06d150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f905c0ce410 0x7f905c14d5e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f905c070990 0x7f905c14dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 --2- 172.21.15.33:0/2243791361 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f905c0ce030 0x7f905c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 >> 172.21.15.33:0/2243791361 conn(0x7f905c072230 msgr2=0x7f905c14f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 shutdown_connections 2024-05-08T23:32:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:21.888+0000 7f90615db700 1 -- 172.21.15.33:0/2243791361 wait complete. 2024-05-08T23:32:21.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:08.545052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:09.301342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:21.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.042+0000 7fc1557a8700 1 Processor -- start 2024-05-08T23:32:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.042+0000 7fc1557a8700 1 -- start start 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc1557a8700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1500712f0 0x7fc15015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc1557a8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc15006d980 0x7fc15015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc1557a8700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1500c2860 con 0x7fc150154540 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc1557a8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1500c3540 con 0x7fc15006d980 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc1557a8700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1500c8620 con 0x7fc1500712f0 2024-05-08T23:32:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc14f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc15006d980 0x7fc15015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc14f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc15006d980 0x7fc15015de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53154/0 (socket says 172.21.15.33:53154) 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.043+0000 7fc14f7fe700 1 -- 172.21.15.33:0/2967890662 learned_addr learned my addr 172.21.15.33:0/2967890662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14e7fc700 1 --2- 172.21.15.33:0/2967890662 >> v2:172.21.15.42:3300/0 conn(0x7fc1500712f0 0x7fc15015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14f7fe700 1 -- 172.21.15.33:0/2967890662 >> v2:172.21.15.42:3300/0 conn(0x7fc1500712f0 msgr2=0x7fc15015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14f7fe700 1 --2- 172.21.15.33:0/2967890662 >> v2:172.21.15.42:3300/0 conn(0x7fc1500712f0 0x7fc15015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14f7fe700 1 -- 172.21.15.33:0/2967890662 >> v1:172.21.15.42:6789/0 conn(0x7fc150154540 legacy=0x7fc1500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14f7fe700 1 -- 172.21.15.33:0/2967890662 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500c2ae0 con 0x7fc15006d980 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14f7fe700 1 --2- 172.21.15.33:0/2967890662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc15006d980 0x7fc15015de30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc15006e810 tx=0x7fc14800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf97bb86a6e8263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14dffb700 1 -- 172.21.15.33:0/2967890662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800e070 con 0x7fc15006d980 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14dffb700 1 -- 172.21.15.33:0/2967890662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14800b9c0 con 0x7fc15006d980 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.044+0000 7fc14dffb700 1 -- 172.21.15.33:0/2967890662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800bd70 con 0x7fc15006d980 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 -- 172.21.15.33:0/2967890662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc15006d980 msgr2=0x7fc15015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 --2- 172.21.15.33:0/2967890662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc15006d980 0x7fc15015de30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc15006e810 tx=0x7fc14800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 -- 172.21.15.33:0/2967890662 shutdown_connections 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 --2- 172.21.15.33:0/2967890662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc15006d980 0x7fc15015de30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 --2- 172.21.15.33:0/2967890662 >> v2:172.21.15.42:3300/0 conn(0x7fc1500712f0 0x7fc15015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 -- 172.21.15.33:0/2967890662 >> 172.21.15.33:0/2967890662 conn(0x7fc15006ce60 msgr2=0x7fc15006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 -- 172.21.15.33:0/2967890662 shutdown_connections 2024-05-08T23:32:22.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.045+0000 7fc1557a8700 1 -- 172.21.15.33:0/2967890662 wait complete. 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.047+0000 7fc1557a8700 1 Processor -- start 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.047+0000 7fc1557a8700 1 -- start start 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc15006d980 0x7fc1501f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1500712f0 0x7fc1500c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1500af3c0 con 0x7fc150154540 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501f3ac0 con 0x7fc15006d980 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc1557a8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1500c5a70 con 0x7fc1500712f0 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53164/0 (socket says 172.21.15.33:53164) 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 -- 172.21.15.33:0/3411238937 learned_addr learned my addr 172.21.15.33:0/3411238937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14effd700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc15006d980 0x7fc1501f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14e7fc700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1500712f0 0x7fc1500c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 -- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1500712f0 msgr2=0x7fc1500c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1500712f0 0x7fc1500c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 -- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc15006d980 msgr2=0x7fc1501f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc15006d980 0x7fc1501f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 -- 172.21.15.33:0/3411238937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009b30 con 0x7fc150154540 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.048+0000 7fc14f7fe700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1500747b0 tx=0x7fc148004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.049+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148004880 con 0x7fc150154540 2024-05-08T23:32:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.049+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1480099b0 con 0x7fc150154540 2024-05-08T23:32:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.049+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1500c3fa0 con 0x7fc150154540 2024-05-08T23:32:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.049+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801c680 con 0x7fc150154540 2024-05-08T23:32:22.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.049+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1500c4560 con 0x7fc150154540 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.050+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc150073770 con 0x7fc150154540 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.050+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc14801c7e0 con 0x7fc150154540 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.051+0000 7fc1477fe700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 0x7fc13006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.051+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc14808abb0 con 0x7fc150154540 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.051+0000 7fc14effd700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 0x7fc13006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.051+0000 7fc14effd700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 0x7fc13006d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc138007fd0 tx=0x7fc138009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.055+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc148055d80 con 0x7fc150154540 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.273+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc150075460 con 0x7fc150154540 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.273+0000 7fc1477fe700 1 -- 172.21.15.33:0/3411238937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1480593a0 con 0x7fc150154540 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.275+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 msgr2=0x7fc13006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.275+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 0x7fc13006d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc138007fd0 tx=0x7fc138009040 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.275+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 msgr2=0x7fc1500c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.275+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1500747b0 tx=0x7fc148004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 shutdown_connections 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc13006ac90 0x7fc13006d150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc150154540 0x7fc1500c6a30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc15006d980 0x7fc1501f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 --2- 172.21.15.33:0/3411238937 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1500712f0 0x7fc1500c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 >> 172.21.15.33:0/3411238937 conn(0x7fc15006ce60 msgr2=0x7fc15014f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 shutdown_connections 2024-05-08T23:32:22.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.276+0000 7fc1557a8700 1 -- 172.21.15.33:0/3411238937 wait complete. 2024-05-08T23:32:22.290 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.430+0000 7f6d7d1a2700 1 Processor -- start 2024-05-08T23:32:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.430+0000 7f6d7d1a2700 1 -- start start 2024-05-08T23:32:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7d1a2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d780c8fd0 0x7f6d78157980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7d1a2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d78071410 con 0x7f6d780718a0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7d1a2700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d78066c70 con 0x7f6d7806ef40 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7d1a2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d78157ec0 con 0x7f6d780c8fd0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d76d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6d780c8fd0 0x7f6d78157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d780718a0 0x7f6d78150cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52118/0 (socket says 172.21.15.33:52118) 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d7759e700 1 -- 172.21.15.33:0/1211031269 learned_addr learned my addr 172.21.15.33:0/1211031269 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d76d9d700 1 -- 172.21.15.33:0/1211031269 >> v1:172.21.15.42:6789/0 conn(0x7f6d7806ef40 legacy=0x7f6d780c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d75d9b700 1 -- 172.21.15.33:0/1211031269 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501657360 0 0) 0x7f6d78066c70 con 0x7f6d7806ef40 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d76d9d700 1 -- 172.21.15.33:0/1211031269 >> v1:172.21.15.33:6789/0 conn(0x7f6d780718a0 legacy=0x7f6d78150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d76d9d700 1 -- 172.21.15.33:0/1211031269 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78070820 con 0x7f6d780c8fd0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d76d9d700 1 --2- 172.21.15.33:0/1211031269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780c8fd0 0x7f6d78157980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009840 tx=0x7f6d68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609a220d4bfc2a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d75d9b700 1 -- 172.21.15.33:0/1211031269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68004230 con 0x7f6d780c8fd0 2024-05-08T23:32:22.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.431+0000 7f6d75d9b700 1 -- 172.21.15.33:0/1211031269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d68004390 con 0x7f6d780c8fd0 2024-05-08T23:32:22.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d75d9b700 1 -- 172.21.15.33:0/1211031269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680036a0 con 0x7f6d780c8fd0 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1211031269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780c8fd0 msgr2=0x7f6d78157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1211031269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780c8fd0 0x7f6d78157980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009840 tx=0x7f6d68009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1211031269 shutdown_connections 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1211031269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780c8fd0 0x7f6d78157980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1211031269 >> 172.21.15.33:0/1211031269 conn(0x7f6d7806a990 msgr2=0x7f6d7806ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1211031269 shutdown_connections 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.432+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1211031269 wait complete. 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 Processor -- start 2024-05-08T23:32:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 -- start start 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6d7806ef40 0x7f6d78156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d780c8fd0 0x7f6d781eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780cac80 con 0x7f6d780718a0 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780caf60 con 0x7f6d780c8fd0 2024-05-08T23:32:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.434+0000 7f6d7d1a2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d780cb240 con 0x7f6d7806ef40 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53182/0 (socket says 172.21.15.33:53182) 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 -- 172.21.15.33:0/1158243358 learned_addr learned my addr 172.21.15.33:0/1158243358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7759e700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d780c8fd0 0x7f6d781eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d76d9d700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6d7806ef40 0x7f6d78156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 -- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6d7806ef40 msgr2=0x7f6d78156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6d7806ef40 0x7f6d78156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 -- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d780c8fd0 msgr2=0x7f6d781eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d780c8fd0 0x7f6d781eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 -- 172.21.15.33:0/1158243358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680094a0 con 0x7f6d780718a0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7659c700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c4a0 tx=0x7f6d6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0054e0 con 0x7f6d780718a0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c003680 con 0x7f6d780718a0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d780cb520 con 0x7f6d780718a0 2024-05-08T23:32:22.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c003a30 con 0x7f6d780718a0 2024-05-08T23:32:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.435+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d78156570 con 0x7f6d780718a0 2024-05-08T23:32:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.436+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d780c3c90 con 0x7f6d780718a0 2024-05-08T23:32:22.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.437+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6d6c003b90 con 0x7f6d780718a0 2024-05-08T23:32:22.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.438+0000 7f6d5f7fe700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 0x7f6d6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.438+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d6c088be0 con 0x7f6d780718a0 2024-05-08T23:32:22.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.438+0000 7f6d76d9d700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 0x7f6d6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.438+0000 7f6d76d9d700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 0x7f6d6406d150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6d680090b0 tx=0x7f6d68013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.442+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6d6c053ea0 con 0x7f6d780718a0 2024-05-08T23:32:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.656+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d781eec00 con 0x7f6d780718a0 2024-05-08T23:32:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.657+0000 7f6d5f7fe700 1 -- 172.21.15.33:0/1158243358 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d6c017020 con 0x7f6d780718a0 2024-05-08T23:32:22.657 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 msgr2=0x7f6d6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 0x7f6d6406d150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6d680090b0 tx=0x7f6d68013040 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 msgr2=0x7f6d7815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c4a0 tx=0x7f6d6c00c860 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 shutdown_connections 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6d6406ac90 0x7f6d6406d150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d780c8fd0 0x7f6d781eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6d7806ef40 0x7f6d78156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 --2- 172.21.15.33:0/1158243358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d780718a0 0x7f6d7815a940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 >> 172.21.15.33:0/1158243358 conn(0x7f6d7806a990 msgr2=0x7f6d78153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 shutdown_connections 2024-05-08T23:32:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.659+0000 7f6d7d1a2700 1 -- 172.21.15.33:0/1158243358 wait complete. 2024-05-08T23:32:22.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.812+0000 7f5b5dad4700 1 Processor -- start 2024-05-08T23:32:22.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.812+0000 7f5b5dad4700 1 -- start start 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5b581542f0 0x7f5b580cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581546d0 0x7f5b580ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b580c93c0 0x7f5b580c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5815b140 con 0x7f5b581546d0 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580c9d70 con 0x7f5b581542f0 2024-05-08T23:32:22.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b5dad4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5815a5c0 con 0x7f5b580c93c0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b56ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581546d0 0x7f5b580ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b56ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b581546d0 0x7f5b580ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53194/0 (socket says 172.21.15.33:53194) 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b56ffd700 1 -- 172.21.15.33:0/1742330199 learned_addr learned my addr 172.21.15.33:0/1742330199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b577fe700 1 --2- 172.21.15.33:0/1742330199 >> v2:172.21.15.42:3300/0 conn(0x7f5b581542f0 0x7f5b580cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b57fff700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b580c93c0 0x7f5b580c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b56ffd700 1 -- 172.21.15.33:0/1742330199 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b580c93c0 msgr2=0x7f5b580c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.813+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b580c93c0 0x7f5b580c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b56ffd700 1 -- 172.21.15.33:0/1742330199 >> v2:172.21.15.42:3300/0 conn(0x7f5b581542f0 msgr2=0x7f5b580cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/1742330199 >> v2:172.21.15.42:3300/0 conn(0x7f5b581542f0 0x7f5b580cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b56ffd700 1 -- 172.21.15.33:0/1742330199 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5815a8a0 con 0x7f5b581546d0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581546d0 0x7f5b580ce270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b58075740 tx=0x7f5b4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49c8d63b19dc10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b567fc700 1 -- 172.21.15.33:0/1742330199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011830 con 0x7f5b581546d0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b567fc700 1 -- 172.21.15.33:0/1742330199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c011990 con 0x7f5b581546d0 2024-05-08T23:32:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b567fc700 1 -- 172.21.15.33:0/1742330199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c011d20 con 0x7f5b581546d0 2024-05-08T23:32:22.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b5dad4700 1 -- 172.21.15.33:0/1742330199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581546d0 msgr2=0x7f5b580ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.814+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581546d0 0x7f5b580ce270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b58075740 tx=0x7f5b4c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 -- 172.21.15.33:0/1742330199 shutdown_connections 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b580c93c0 0x7f5b580c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/1742330199 >> v2:172.21.15.42:3300/0 conn(0x7f5b581542f0 0x7f5b580cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/1742330199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b581546d0 0x7f5b580ce270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 -- 172.21.15.33:0/1742330199 >> 172.21.15.33:0/1742330199 conn(0x7f5b58072230 msgr2=0x7f5b58072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 -- 172.21.15.33:0/1742330199 shutdown_connections 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.815+0000 7f5b5dad4700 1 -- 172.21.15.33:0/1742330199 wait complete. 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.816+0000 7f5b5dad4700 1 Processor -- start 2024-05-08T23:32:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.817+0000 7f5b5dad4700 1 -- start start 2024-05-08T23:32:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.817+0000 7f5b5dad4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.817+0000 7f5b5dad4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 0x7f5b580caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b581546d0 0x7f5b581efe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581f3d00 con 0x7f5b580c93c0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581ef980 con 0x7f5b581542f0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580c5cb0 con 0x7f5b581546d0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53210/0 (socket says 172.21.15.33:53210) 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 -- 172.21.15.33:0/206790027 learned_addr learned my addr 172.21.15.33:0/206790027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 0x7f5b580caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b57fff700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b581546d0 0x7f5b581efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 -- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b581546d0 msgr2=0x7f5b581efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b581546d0 0x7f5b581efe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 -- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 msgr2=0x7f5b580caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 0x7f5b580caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 -- 172.21.15.33:0/206790027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c009550 con 0x7f5b580c93c0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 0x7f5b580caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b577fe700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9b0 tx=0x7f5b4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48009710 con 0x7f5b580c93c0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800c850 con 0x7f5b580c93c0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b581f3760 con 0x7f5b580c93c0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800cbc0 con 0x7f5b580c93c0 2024-05-08T23:32:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.818+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b58155210 con 0x7f5b580c93c0 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.820+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58075460 con 0x7f5b580c93c0 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.820+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5b4800cd20 con 0x7f5b580c93c0 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.820+0000 7f5b3ffff700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 0x7f5b4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.820+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b480897f0 con 0x7f5b580c93c0 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.820+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 0x7f5b4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.821+0000 7f5b56ffd700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 0x7f5b4406d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b58151840 tx=0x7f5b4c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:22.825+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f5b480549d0 con 0x7f5b580c93c0 2024-05-08T23:32:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.042+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b581f3a20 con 0x7f5b580c93c0 2024-05-08T23:32:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.043+0000 7f5b3ffff700 1 -- 172.21.15.33:0/206790027 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b48021020 con 0x7f5b580c93c0 2024-05-08T23:32:23.043 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T23:32:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 msgr2=0x7f5b4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 0x7f5b4406d150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b58151840 tx=0x7f5b4c008030 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 msgr2=0x7f5b580c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9b0 tx=0x7f5b4800ed70 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 shutdown_connections 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5b4406ac90 0x7f5b4406d150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5b581546d0 0x7f5b581efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b580c93c0 0x7f5b580c4700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 --2- 172.21.15.33:0/206790027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b581542f0 0x7f5b580caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 >> 172.21.15.33:0/206790027 conn(0x7f5b58072230 msgr2=0x7f5b58072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 shutdown_connections 2024-05-08T23:32:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.045+0000 7f5b5dad4700 1 -- 172.21.15.33:0/206790027 wait complete. 2024-05-08T23:32:23.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.194+0000 7fb211839700 1 Processor -- start 2024-05-08T23:32:23.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.194+0000 7fb211839700 1 -- start start 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb211839700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb20c154540 0x7fb20c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb211839700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb20c06d980 0x7fb20c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb211839700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c0c2860 con 0x7fb20c0712f0 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb211839700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c0c3540 con 0x7fb20c06d980 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb211839700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c0c9600 con 0x7fb20c154540 2024-05-08T23:32:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.195+0000 7fb20b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb20c06d980 0x7fb20c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb20c06d980 0x7fb20c15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53222/0 (socket says 172.21.15.33:53222) 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 -- 172.21.15.33:0/253331897 learned_addr learned my addr 172.21.15.33:0/253331897 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20affd700 1 --2- 172.21.15.33:0/253331897 >> v2:172.21.15.42:3300/0 conn(0x7fb20c154540 0x7fb20c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 -- 172.21.15.33:0/253331897 >> v2:172.21.15.42:3300/0 conn(0x7fb20c154540 msgr2=0x7fb20c0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 --2- 172.21.15.33:0/253331897 >> v2:172.21.15.42:3300/0 conn(0x7fb20c154540 0x7fb20c0c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 -- 172.21.15.33:0/253331897 >> v1:172.21.15.42:6789/0 conn(0x7fb20c0712f0 legacy=0x7fb20c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 -- 172.21.15.33:0/253331897 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0c8620 con 0x7fb20c06d980 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb20b7fe700 1 --2- 172.21.15.33:0/253331897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15e000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb20c04f2b0 tx=0x7fb20400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5e6924d71a1d90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb209ffb700 1 -- 172.21.15.33:0/253331897 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400e070 con 0x7fb20c06d980 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb209ffb700 1 -- 172.21.15.33:0/253331897 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20400b910 con 0x7fb20c06d980 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.196+0000 7fb209ffb700 1 -- 172.21.15.33:0/253331897 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20400bcc0 con 0x7fb20c06d980 2024-05-08T23:32:23.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 -- 172.21.15.33:0/253331897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 msgr2=0x7fb20c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 --2- 172.21.15.33:0/253331897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15e000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb20c04f2b0 tx=0x7fb20400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 -- 172.21.15.33:0/253331897 shutdown_connections 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 --2- 172.21.15.33:0/253331897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15e000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 --2- 172.21.15.33:0/253331897 >> v2:172.21.15.42:3300/0 conn(0x7fb20c154540 0x7fb20c0c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 -- 172.21.15.33:0/253331897 >> 172.21.15.33:0/253331897 conn(0x7fb20c06ce60 msgr2=0x7fb20c06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 -- 172.21.15.33:0/253331897 shutdown_connections 2024-05-08T23:32:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.197+0000 7fb211839700 1 -- 172.21.15.33:0/253331897 wait complete. 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.198+0000 7fb211839700 1 Processor -- start 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.198+0000 7fb211839700 1 -- start start 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb20c0712f0 0x7fb20c14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c154540 0x7fb20c06ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0c5120 con 0x7fb20c06d980 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c15c640 con 0x7fb20c154540 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb211839700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c15bc70 con 0x7fb20c0712f0 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb20affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb20affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53224/0 (socket says 172.21.15.33:53224) 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb20affd700 1 -- 172.21.15.33:0/2408042221 learned_addr learned my addr 172.21.15.33:0/2408042221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb20a7fc700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb20c0712f0 0x7fb20c14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.199+0000 7fb20b7fe700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c154540 0x7fb20c06ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 -- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb20c0712f0 msgr2=0x7fb20c14c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb20c0712f0 0x7fb20c14c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 -- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c154540 msgr2=0x7fb20c06ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c154540 0x7fb20c06ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 -- 172.21.15.33:0/2408042221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204009b30 con 0x7fb20c06d980 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb20affd700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e8a0 tx=0x7fb20000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200009590 con 0x7fb20c06d980 2024-05-08T23:32:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200013680 con 0x7fb20c06d980 2024-05-08T23:32:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c15e570 con 0x7fb20c06d980 2024-05-08T23:32:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2000139f0 con 0x7fb20c06d980 2024-05-08T23:32:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.200+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c15cd10 con 0x7fb20c06d980 2024-05-08T23:32:23.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.201+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c073770 con 0x7fb20c06d980 2024-05-08T23:32:23.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.202+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb200013b50 con 0x7fb20c06d980 2024-05-08T23:32:23.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.202+0000 7fb1f3fff700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 0x7fb1f806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.202+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb200089070 con 0x7fb20c06d980 2024-05-08T23:32:23.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.202+0000 7fb20a7fc700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 0x7fb1f806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.203+0000 7fb20a7fc700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 0x7fb1f806d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb20c071c40 tx=0x7fb1fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.207+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb2000542e0 con 0x7fb20c06d980 2024-05-08T23:32:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.478+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb20c075460 con 0x7fb20c06d980 2024-05-08T23:32:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.479+0000 7fb1f3fff700 1 -- 172.21.15.33:0/2408042221 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7fb200057900 con 0x7fb20c06d980 2024-05-08T23:32:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:23.479 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 msgr2=0x7fb1f806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 0x7fb1f806d180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb20c071c40 tx=0x7fb1fc009040 comp rx=0 tx=0).stop 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 msgr2=0x7fb20c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e8a0 tx=0x7fb20000ebb0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 shutdown_connections 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb1f806acc0 0x7fb1f806d180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb20c154540 0x7fb20c06ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb20c06d980 0x7fb20c15c100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 --2- 172.21.15.33:0/2408042221 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb20c0712f0 0x7fb20c14c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.481+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 >> 172.21.15.33:0/2408042221 conn(0x7fb20c06ce60 msgr2=0x7fb20c14eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.482+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 shutdown_connections 2024-05-08T23:32:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.482+0000 7fb211839700 1 -- 172.21.15.33:0/2408042221 wait complete. 2024-05-08T23:32:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:32:23.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:23.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.631+0000 7ff79a65b700 1 Processor -- start 2024-05-08T23:32:23.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.631+0000 7ff79a65b700 1 -- start start 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff79a65b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff79a65b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7940cbbe0 0x7ff79415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff79a65b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79415b350 con 0x7ff794154970 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff79a65b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79415e8a0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff79a65b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79415a5d0 con 0x7ff7940caad0 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7940cbbe0 0x7ff79415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7940cbbe0 0x7ff79415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53230/0 (socket says 172.21.15.33:53230) 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 -- 172.21.15.33:0/3140044551 learned_addr learned my addr 172.21.15.33:0/3140044551 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff7937fe700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 -- 172.21.15.33:0/3140044551 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 msgr2=0x7ff7940cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 -- 172.21.15.33:0/3140044551 >> v1:172.21.15.42:6789/0 conn(0x7ff794154970 legacy=0x7ff79406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.632+0000 7ff798bf8700 1 -- 172.21.15.33:0/3140044551 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79415e5c0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff798bf8700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff79415d9a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009e50 tx=0x7ff78c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07a5d08e0fde5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff792ffd700 1 -- 172.21.15.33:0/3140044551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00e070 con 0x7ff7940cbbe0 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff792ffd700 1 -- 172.21.15.33:0/3140044551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00b910 con 0x7ff7940cbbe0 2024-05-08T23:32:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff792ffd700 1 -- 172.21.15.33:0/3140044551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00bcc0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 -- 172.21.15.33:0/3140044551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 msgr2=0x7ff79415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff79415d9a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009e50 tx=0x7ff78c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 -- 172.21.15.33:0/3140044551 shutdown_connections 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff79415d9a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 --2- 172.21.15.33:0/3140044551 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.633+0000 7ff79a65b700 1 -- 172.21.15.33:0/3140044551 >> 172.21.15.33:0/3140044551 conn(0x7ff79406cd80 msgr2=0x7ff79406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.634+0000 7ff79a65b700 1 -- 172.21.15.33:0/3140044551 shutdown_connections 2024-05-08T23:32:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.634+0000 7ff79a65b700 1 -- 172.21.15.33:0/3140044551 wait complete. 2024-05-08T23:32:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.635+0000 7ff79a65b700 1 Processor -- start 2024-05-08T23:32:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.635+0000 7ff79a65b700 1 -- start start 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff794154970 0x7ff7941e4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940c7f30 con 0x7ff7940cbbe0 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940c9d90 con 0x7ff794154970 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff79a65b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940c3f60 con 0x7ff7940caad0 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53240/0 (socket says 172.21.15.33:53240) 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 -- 172.21.15.33:0/2381688705 learned_addr learned my addr 172.21.15.33:0/2381688705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff798bf8700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff794154970 0x7ff7941e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff793fff700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 -- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 msgr2=0x7ff7940c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 -- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff794154970 msgr2=0x7ff7941e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff794154970 0x7ff7941e4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 -- 172.21.15.33:0/2381688705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c009b30 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.636+0000 7ff7937fe700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e980 tx=0x7ff78800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.637+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788009710 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.637+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800c7f0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.637+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7940c6fd0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.637+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800cba0 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.637+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940c7590 con 0x7ff7940cbbe0 2024-05-08T23:32:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.638+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940c8070 con 0x7ff7940cbbe0 2024-05-08T23:32:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.638+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff78800cd00 con 0x7ff7940cbbe0 2024-05-08T23:32:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.639+0000 7ff790ff9700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 0x7ff78006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.639+0000 7ff793fff700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 0x7ff78006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.639+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff788089360 con 0x7ff7940cbbe0 2024-05-08T23:32:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.640+0000 7ff793fff700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 0x7ff78006d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff784000f60 tx=0x7ff784009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.644+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff7940c8070 con 0x7ff7940cbbe0 2024-05-08T23:32:23.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.868+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7940c8070 con 0x7ff7940cbbe0 2024-05-08T23:32:23.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.869+0000 7ff790ff9700 1 -- 172.21.15.33:0/2381688705 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7ff7940c8070 con 0x7ff7940cbbe0 2024-05-08T23:32:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:23.869 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 msgr2=0x7ff78006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 0x7ff78006d150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff784000f60 tx=0x7ff784009040 comp rx=0 tx=0).stop 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 msgr2=0x7ff7940b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e980 tx=0x7ff78800ed40 comp rx=0 tx=0).stop 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 shutdown_connections 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff78006ac90 0x7ff78006d150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff794154970 0x7ff7941e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff7940caad0 0x7ff7940c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 --2- 172.21.15.33:0/2381688705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7940cbbe0 0x7ff7940b3d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.871+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 >> 172.21.15.33:0/2381688705 conn(0x7ff79406cd80 msgr2=0x7ff79406ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.872+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 shutdown_connections 2024-05-08T23:32:23.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:23.872+0000 7ff79a65b700 1 -- 172.21.15.33:0/2381688705 wait complete. 2024-05-08T23:32:23.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:08.545052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:09.301342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:23.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.024+0000 7fa7904b7700 1 Processor -- start 2024-05-08T23:32:24.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.025+0000 7fa7904b7700 1 -- start start 2024-05-08T23:32:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa7904b7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa788071120 0x7fa7880b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa7904b7700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 0x7fa788071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa7904b7700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78806a990 con 0x7fa78815d530 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa7904b7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880719b0 con 0x7fa788071120 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa7904b7700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880b2420 con 0x7fa7880b2010 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa788071120 0x7fa7880b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78e253700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa78815d530 0x7fa7880c9c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52132/0 (socket says 172.21.15.33:52132) 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78e253700 1 -- 172.21.15.33:0/2223012715 learned_addr learned my addr 172.21.15.33:0/2223012715 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78ea54700 1 --2- 172.21.15.33:0/2223012715 >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 0x7fa788071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 -- 172.21.15.33:0/2223012715 >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 msgr2=0x7fa788071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 --2- 172.21.15.33:0/2223012715 >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 0x7fa788071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 -- 172.21.15.33:0/2223012715 >> v1:172.21.15.33:6789/0 conn(0x7fa78815d530 legacy=0x7fa7880c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 -- 172.21.15.33:0/2223012715 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78806cb50 con 0x7fa788071120 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78ea54700 1 --2- 172.21.15.33:0/2223012715 >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 0x7fa788071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78d251700 1 -- 172.21.15.33:0/2223012715 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754565077 0 0) 0x7fa78806a990 con 0x7fa78815d530 2024-05-08T23:32:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78da52700 1 --2- 172.21.15.33:0/2223012715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa788071120 0x7fa7880b1ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7880716b0 tx=0x7fa778009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50284b324a21eceb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78d251700 1 -- 172.21.15.33:0/2223012715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778011830 con 0x7fa788071120 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78d251700 1 -- 172.21.15.33:0/2223012715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778011990 con 0x7fa788071120 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.026+0000 7fa78d251700 1 -- 172.21.15.33:0/2223012715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778011d20 con 0x7fa788071120 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 -- 172.21.15.33:0/2223012715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa788071120 msgr2=0x7fa7880b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 --2- 172.21.15.33:0/2223012715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa788071120 0x7fa7880b1ad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7880716b0 tx=0x7fa778009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 -- 172.21.15.33:0/2223012715 shutdown_connections 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 --2- 172.21.15.33:0/2223012715 >> v2:172.21.15.42:3300/0 conn(0x7fa7880b2010 0x7fa788071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 --2- 172.21.15.33:0/2223012715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa788071120 0x7fa7880b1ad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 -- 172.21.15.33:0/2223012715 >> 172.21.15.33:0/2223012715 conn(0x7fa78806d2b0 msgr2=0x7fa78806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 -- 172.21.15.33:0/2223012715 shutdown_connections 2024-05-08T23:32:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.027+0000 7fa7904b7700 1 -- 172.21.15.33:0/2223012715 wait complete. 2024-05-08T23:32:24.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.028+0000 7fa7904b7700 1 Processor -- start 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.028+0000 7fa7904b7700 1 -- start start 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa788071120 0x7fa7881566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa78815d530 0x7fa788154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880c1480 con 0x7fa7880b2010 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880af3f0 con 0x7fa78815d530 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa7904b7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880b18c0 con 0x7fa788071120 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa78da52700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa78da52700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53252/0 (socket says 172.21.15.33:53252) 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa78da52700 1 -- 172.21.15.33:0/769815333 learned_addr learned my addr 172.21.15.33:0/769815333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa78e253700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa788071120 0x7fa7881566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.029+0000 7fa78ea54700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa78815d530 0x7fa788154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 -- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa788071120 msgr2=0x7fa7881566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa788071120 0x7fa7881566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 -- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa78815d530 msgr2=0x7fa788154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa78815d530 0x7fa788154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 -- 172.21.15.33:0/769815333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780094a0 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa78da52700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa778004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780038a0 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77801f070 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7880b1b60 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778003c40 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.030+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78815c180 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.031+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78815bfe0 con 0x7fa7880b2010 2024-05-08T23:32:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.031+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa778026480 con 0x7fa7880b2010 2024-05-08T23:32:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.032+0000 7fa77effd700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 0x7fa76c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.032+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa778021070 con 0x7fa7880b2010 2024-05-08T23:32:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.032+0000 7fa78e253700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 0x7fa76c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.032+0000 7fa78e253700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 0x7fa76c06d150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa784000f60 tx=0x7fa784007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.037+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa78815bfe0 con 0x7fa7880b2010 2024-05-08T23:32:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.252+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7881578f0 con 0x7fa7880b2010 2024-05-08T23:32:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.252+0000 7fa77effd700 1 -- 172.21.15.33:0/769815333 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7881578f0 con 0x7fa7880b2010 2024-05-08T23:32:24.252 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.254+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 msgr2=0x7fa76c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.254+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 0x7fa76c06d150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa784000f60 tx=0x7fa784007c90 comp rx=0 tx=0).stop 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.254+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 msgr2=0x7fa788157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.254+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa778004750 comp rx=0 tx=0).stop 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 shutdown_connections 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa76c06ac90 0x7fa76c06d150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa78815d530 0x7fa788154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa788071120 0x7fa7881566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 --2- 172.21.15.33:0/769815333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880b2010 0x7fa788157bd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 >> 172.21.15.33:0/769815333 conn(0x7fa78806d2b0 msgr2=0x7fa78815ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 shutdown_connections 2024-05-08T23:32:24.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.255+0000 7fa7904b7700 1 -- 172.21.15.33:0/769815333 wait complete. 2024-05-08T23:32:24.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:24.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.410+0000 7f70278b3700 1 Processor -- start 2024-05-08T23:32:24.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.411+0000 7f70278b3700 1 -- start start 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.411+0000 7f70278b3700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f70200c38a0 0x7f70200c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.411+0000 7f70278b3700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7020070450 con 0x7f7020152350 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.411+0000 7f70278b3700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7020070730 con 0x7f70200c9a60 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.411+0000 7f70278b3700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70200c2e90 con 0x7f70200c38a0 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f702564f700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7020152350 0x7f7020158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52144/0 (socket says 172.21.15.33:52144) 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f702564f700 1 -- 172.21.15.33:0/105135778 learned_addr learned my addr 172.21.15.33:0/105135778 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7024e4e700 1 --2- 172.21.15.33:0/105135778 >> v2:172.21.15.42:3300/0 conn(0x7f70200c38a0 0x7f70200c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756312122 0 0) 0x7f7020070450 con 0x7f7020152350 2024-05-08T23:32:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f700c0035b0 con 0x7f7020152350 2024-05-08T23:32:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2354894028 0 0) 0x7f700c0035b0 con 0x7f7020152350 2024-05-08T23:32:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 >> v2:172.21.15.42:3300/0 conn(0x7f70200c38a0 msgr2=0x7f70200c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 --2- 172.21.15.33:0/105135778 >> v2:172.21.15.42:3300/0 conn(0x7f70200c38a0 0x7f70200c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 >> v1:172.21.15.42:6789/0 conn(0x7f70200c9a60 legacy=0x7f70200c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7020070a10 con 0x7f7020152350 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f7010001670 con 0x7f7020152350 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 >> v1:172.21.15.33:6789/0 conn(0x7f7020152350 legacy=0x7f7020158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 0x7f700c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c005210 0x7f700c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200c2e90 con 0x7f700c003a00 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00bfd0 con 0x7f700c004850 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c00c190 con 0x7f700c005210 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f702564f700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 0x7f700c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.412+0000 7f7024e4e700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f7025e50700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c005210 0x7f700c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 -- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c005210 msgr2=0x7f700c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c005210 0x7f700c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 -- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f700c004850 msgr2=0x7f700c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f700c004850 0x7f700c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 -- 172.21.15.33:0/105135778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0035b0 con 0x7f700c003a00 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f7025e50700 1 -- 172.21.15.33:0/105135778 reap_dead start 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f702564f700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 0x7f700c003de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f701000c210 tx=0x7f700c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1126166b727dde1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010007b70 con 0x7f700c003a00 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701000ccb0 con 0x7f700c003a00 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.413+0000 7f7017fff700 1 -- 172.21.15.33:0/105135778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010005ba0 con 0x7f700c003a00 2024-05-08T23:32:24.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 -- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 msgr2=0x7f700c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 0x7f700c003de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f701000c210 tx=0x7f700c005880 comp rx=0 tx=0).stop 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 -- 172.21.15.33:0/105135778 shutdown_connections 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 --2- 172.21.15.33:0/105135778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f700c003a00 0x7f700c003de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 -- 172.21.15.33:0/105135778 >> 172.21.15.33:0/105135778 conn(0x7f70200666c0 msgr2=0x7f7020066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 -- 172.21.15.33:0/105135778 shutdown_connections 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.414+0000 7f70278b3700 1 -- 172.21.15.33:0/105135778 wait complete. 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.415+0000 7f70278b3700 1 Processor -- start 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.415+0000 7f70278b3700 1 -- start start 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c003a00 0x7f70200c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020148d00 0x7f70201f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200cdf60 con 0x7f7020146690 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020147060 con 0x7f7020148d00 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f70278b3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702014f830 con 0x7f700c003a00 2024-05-08T23:32:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53270/0 (socket says 172.21.15.33:53270) 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 -- 172.21.15.33:0/3981138904 learned_addr learned my addr 172.21.15.33:0/3981138904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f702564f700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c003a00 0x7f70200c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7024e4e700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020148d00 0x7f70201f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 -- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c003a00 msgr2=0x7f70200c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c003a00 0x7f70200c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 -- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020148d00 msgr2=0x7f70201f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020148d00 0x7f70201f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 -- 172.21.15.33:0/3981138904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7020148870 con 0x7f7020146690 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.416+0000 7f7025e50700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f70200ce5b0 tx=0x7f701800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.417+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800fe70 con 0x7f7020146690 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.417+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7018003c10 con 0x7f7020146690 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.417+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7020155b00 con 0x7f7020146690 2024-05-08T23:32:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.417+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701800d990 con 0x7f7020146690 2024-05-08T23:32:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.417+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70201f1260 con 0x7f7020146690 2024-05-08T23:32:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.418+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020148a00 con 0x7f7020146690 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.418+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f701800daf0 con 0x7f7020146690 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.419+0000 7f7015ffb700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 0x7f700406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.419+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7018089500 con 0x7f7020146690 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.419+0000 7f702564f700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 0x7f700406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.419+0000 7f702564f700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 0x7f700406d150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7010000c00 tx=0x7f7010007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.423+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7018054770 con 0x7f7020146690 2024-05-08T23:32:24.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.639+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7020075460 con 0x7f7020146690 2024-05-08T23:32:24.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.639+0000 7f7015ffb700 1 -- 172.21.15.33:0/3981138904 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7018057d90 con 0x7f7020146690 2024-05-08T23:32:24.640 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 msgr2=0x7f700406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 0x7f700406d150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7010000c00 tx=0x7f7010007000 comp rx=0 tx=0).stop 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 msgr2=0x7f7020146b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f70200ce5b0 tx=0x7f701800d430 comp rx=0 tx=0).stop 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 shutdown_connections 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f700406ac90 0x7f700406d150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7020146690 0x7f7020146b20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f700c003a00 0x7f70200c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 --2- 172.21.15.33:0/3981138904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020148d00 0x7f70201f0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 >> 172.21.15.33:0/3981138904 conn(0x7f70200666c0 msgr2=0x7f70200ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 shutdown_connections 2024-05-08T23:32:24.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.642+0000 7f70278b3700 1 -- 172.21.15.33:0/3981138904 wait complete. 2024-05-08T23:32:24.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:24.710 INFO:tasks.ceph.mds.d.smithi042.stderr:2024-05-08T23:32:24.711+0000 7f0509067700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:32:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.796+0000 7fdc87c4f700 1 Processor -- start 2024-05-08T23:32:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.796+0000 7fdc87c4f700 1 -- start start 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc87c4f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdc80072890 0x7fdc801626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc87c4f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8006a990 con 0x7fdc800c6e20 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc87c4f700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8006abe0 con 0x7fdc80158a70 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc87c4f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8006c620 con 0x7fdc80072890 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc851ea700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdc800c6e20 0x7fdc8015e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52156/0 (socket says 172.21.15.33:52156) 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc851ea700 1 -- 172.21.15.33:0/4228312915 learned_addr learned my addr 172.21.15.33:0/4228312915 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc861ec700 1 --2- 172.21.15.33:0/4228312915 >> v2:172.21.15.42:3300/0 conn(0x7fdc80072890 0x7fdc801626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381539417 0 0) 0x7fdc8006a990 con 0x7fdc800c6e20 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc6c0035b0 con 0x7fdc800c6e20 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528439356 0 0) 0x7fdc8006abe0 con 0x7fdc80158a70 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8006a990 con 0x7fdc80158a70 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc861ec700 1 -- 172.21.15.33:0/4228312915 >> v1:172.21.15.42:6789/0 conn(0x7fdc80158a70 legacy=0x7fdc80072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc861ec700 1 -- 172.21.15.33:0/4228312915 >> v1:172.21.15.33:6789/0 conn(0x7fdc800c6e20 legacy=0x7fdc8015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.797+0000 7fdc861ec700 1 -- 172.21.15.33:0/4228312915 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8006c900 con 0x7fdc80072890 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc861ec700 1 --2- 172.21.15.33:0/4228312915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80072890 0x7fdc801626f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009dc0 tx=0x7fdc7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec05987d16a9cf5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800e070 con 0x7fdc80072890 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7800bae0 con 0x7fdc80072890 2024-05-08T23:32:24.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc849e9700 1 -- 172.21.15.33:0/4228312915 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800be90 con 0x7fdc80072890 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc87c4f700 1 -- 172.21.15.33:0/4228312915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80072890 msgr2=0x7fdc801626f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.798+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/4228312915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80072890 0x7fdc801626f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009dc0 tx=0x7fdc7800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.799+0000 7fdc87c4f700 1 -- 172.21.15.33:0/4228312915 shutdown_connections 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.799+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/4228312915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80072890 0x7fdc801626f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.799+0000 7fdc87c4f700 1 -- 172.21.15.33:0/4228312915 >> 172.21.15.33:0/4228312915 conn(0x7fdc8006d2b0 msgr2=0x7fdc8006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.799+0000 7fdc87c4f700 1 -- 172.21.15.33:0/4228312915 shutdown_connections 2024-05-08T23:32:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.799+0000 7fdc87c4f700 1 -- 172.21.15.33:0/4228312915 wait complete. 2024-05-08T23:32:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.800+0000 7fdc87c4f700 1 Processor -- start 2024-05-08T23:32:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.800+0000 7fdc87c4f700 1 -- start start 2024-05-08T23:32:24.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 0x7fdc801f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80158a70 0x7fdc80153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801551c0 con 0x7fdc800c6e20 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80152b10 con 0x7fdc80158a70 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc87c4f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015ccb0 con 0x7fdc80072890 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53276/0 (socket says 172.21.15.33:53276) 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 -- 172.21.15.33:0/409649459 learned_addr learned my addr 172.21.15.33:0/409649459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc861ec700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80158a70 0x7fdc80153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc859eb700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 0x7fdc801f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 -- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 msgr2=0x7fdc801f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 0x7fdc801f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 -- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80158a70 msgr2=0x7fdc80153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80158a70 0x7fdc80153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 -- 172.21.15.33:0/409649459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009a70 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc859eb700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 0x7fdc801f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.801+0000 7fdc851ea700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005910 tx=0x7fdc7000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.802+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70005d70 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.802+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc700070d0 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.802+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc80155410 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.802+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70007480 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.802+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc80152d60 con 0x7fdc800c6e20 2024-05-08T23:32:24.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.803+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc80075460 con 0x7fdc800c6e20 2024-05-08T23:32:24.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.803+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdc700075e0 con 0x7fdc800c6e20 2024-05-08T23:32:24.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.804+0000 7fdc767fc700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 0x7fdc6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.804+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc700887f0 con 0x7fdc800c6e20 2024-05-08T23:32:24.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.804+0000 7fdc859eb700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 0x7fdc6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.804+0000 7fdc859eb700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 0x7fdc6406d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0011e0 tx=0x7fdc7c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:24.809+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdc700539d0 con 0x7fdc800c6e20 2024-05-08T23:32:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.023+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc800c97b0 con 0x7fdc800c6e20 2024-05-08T23:32:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.024+0000 7fdc767fc700 1 -- 172.21.15.33:0/409649459 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc70017020 con 0x7fdc800c6e20 2024-05-08T23:32:25.024 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 msgr2=0x7fdc6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 0x7fdc6406d150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0011e0 tx=0x7fdc7c009040 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 msgr2=0x7fdc80153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005910 tx=0x7fdc7000c9e0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 shutdown_connections 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdc6406ac90 0x7fdc6406d150 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc80158a70 0x7fdc80153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdc80072890 0x7fdc801f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 --2- 172.21.15.33:0/409649459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc800c6e20 0x7fdc80153560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 >> 172.21.15.33:0/409649459 conn(0x7fdc8006d2b0 msgr2=0x7fdc801624f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 shutdown_connections 2024-05-08T23:32:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.026+0000 7fdc87c4f700 1 -- 172.21.15.33:0/409649459 wait complete. 2024-05-08T23:32:25.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.176+0000 7fc1d2d8e700 1 Processor -- start 2024-05-08T23:32:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.176+0000 7fc1d2d8e700 1 -- start start 2024-05-08T23:32:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.177+0000 7fc1d2d8e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1cc072270 0x7fc1cc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.177+0000 7fc1d2d8e700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1cc0710b0 0x7fc1cc0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d2d8e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1cc06a990 con 0x7fc1cc154550 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d2d8e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc06b7a0 con 0x7fc1cc072270 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d2d8e700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc06cb50 con 0x7fc1cc0710b0 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc1cc072270 0x7fc1cc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d0b2a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc1cc154550 0x7fc1cc15a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52172/0 (socket says 172.21.15.33:52172) 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d0b2a700 1 -- 172.21.15.33:0/2921148154 learned_addr learned my addr 172.21.15.33:0/2921148154 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1d132b700 1 --2- 172.21.15.33:0/2921148154 >> v2:172.21.15.42:3300/0 conn(0x7fc1cc0710b0 0x7fc1cc0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 -- 172.21.15.33:0/2921148154 >> v2:172.21.15.42:3300/0 conn(0x7fc1cc0710b0 msgr2=0x7fc1cc0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 --2- 172.21.15.33:0/2921148154 >> v2:172.21.15.42:3300/0 conn(0x7fc1cc0710b0 0x7fc1cc0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 -- 172.21.15.33:0/2921148154 >> v1:172.21.15.33:6789/0 conn(0x7fc1cc154550 legacy=0x7fc1cc15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 -- 172.21.15.33:0/2921148154 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc071de0 con 0x7fc1cc072270 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cb7fe700 1 -- 172.21.15.33:0/2921148154 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188727681 0 0) 0x7fc1cc06a990 con 0x7fc1cc154550 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cbfff700 1 --2- 172.21.15.33:0/2921148154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc072270 0x7fc1cc06c610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc157f20 tx=0x7fc1b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b0bf5f4d063b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cb7fe700 1 -- 172.21.15.33:0/2921148154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4011830 con 0x7fc1cc072270 2024-05-08T23:32:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cb7fe700 1 -- 172.21.15.33:0/2921148154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b4011990 con 0x7fc1cc072270 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.178+0000 7fc1cb7fe700 1 -- 172.21.15.33:0/2921148154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4011d20 con 0x7fc1cc072270 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/2921148154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc072270 msgr2=0x7fc1cc06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/2921148154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc072270 0x7fc1cc06c610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc157f20 tx=0x7fc1b4009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/2921148154 shutdown_connections 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/2921148154 >> v2:172.21.15.42:3300/0 conn(0x7fc1cc0710b0 0x7fc1cc0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/2921148154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc072270 0x7fc1cc06c610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/2921148154 >> 172.21.15.33:0/2921148154 conn(0x7fc1cc0ccc50 msgr2=0x7fc1cc0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/2921148154 shutdown_connections 2024-05-08T23:32:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.179+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/2921148154 wait complete. 2024-05-08T23:32:25.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.180+0000 7fc1d2d8e700 1 Processor -- start 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.180+0000 7fc1d2d8e700 1 -- start start 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1cc072270 0x7fc1cc1f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1cc154550 0x7fc1cc152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0c46b0 con 0x7fc1cc0710b0 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1eff60 con 0x7fc1cc154550 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d2d8e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1f4420 con 0x7fc1cc072270 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d0b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d0b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53296/0 (socket says 172.21.15.33:53296) 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.181+0000 7fc1d0b2a700 1 -- 172.21.15.33:0/1703726096 learned_addr learned my addr 172.21.15.33:0/1703726096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:25.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d132b700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1cc154550 0x7fc1cc152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1cbfff700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1cc072270 0x7fc1cc1f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 -- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1cc072270 msgr2=0x7fc1cc1f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1cc072270 0x7fc1cc1f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 -- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1cc154550 msgr2=0x7fc1cc152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1cc154550 0x7fc1cc152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 -- 172.21.15.33:0/1703726096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40094a0 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d0b2a700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0011e0 tx=0x7fc1bc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004c80 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc004e00 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc1f0500 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007310 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.182+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc0c48e0 con 0x7fc1cc0710b0 2024-05-08T23:32:25.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.183+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc075460 con 0x7fc1cc0710b0 2024-05-08T23:32:25.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.184+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1bc007470 con 0x7fc1cc0710b0 2024-05-08T23:32:25.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.184+0000 7fc1c97fa700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 0x7fc1ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.184+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1bc089010 con 0x7fc1cc0710b0 2024-05-08T23:32:25.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.184+0000 7fc1cbfff700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 0x7fc1ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.185+0000 7fc1cbfff700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 0x7fc1ac06d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc157f20 tx=0x7fc1b4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.189+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc1bc054150 con 0x7fc1cc0710b0 2024-05-08T23:32:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.457+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1cc0c3d70 con 0x7fc1cc0710b0 2024-05-08T23:32:25.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.457+0000 7fc1c97fa700 1 -- 172.21.15.33:0/1703726096 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7fc1bc057770 con 0x7fc1cc0710b0 2024-05-08T23:32:25.458 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:25.458 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.459+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 msgr2=0x7fc1ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.459+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 0x7fc1ac06d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc157f20 tx=0x7fc1b4008030 comp rx=0 tx=0).stop 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.459+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 msgr2=0x7fc1cc1f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.459+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0011e0 tx=0x7fc1bc00ca30 comp rx=0 tx=0).stop 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 shutdown_connections 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc1ac06ac90 0x7fc1ac06d150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1cc154550 0x7fc1cc152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1cc0710b0 0x7fc1cc1f3ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 --2- 172.21.15.33:0/1703726096 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1cc072270 0x7fc1cc1f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 >> 172.21.15.33:0/1703726096 conn(0x7fc1cc0ccc50 msgr2=0x7fc1cc0cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 shutdown_connections 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.460+0000 7fc1d2d8e700 1 -- 172.21.15.33:0/1703726096 wait complete. 2024-05-08T23:32:25.461 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:32:25.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.609+0000 7f0c13666700 1 Processor -- start 2024-05-08T23:32:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.609+0000 7f0c13666700 1 -- start start 2024-05-08T23:32:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.610+0000 7f0c13666700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c1542f0 0x7f0c0c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.610+0000 7f0c13666700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0c0c9ad0 0x7f0c0c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.610+0000 7f0c13666700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c15b140 con 0x7f0c0c1546d0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.610+0000 7f0c13666700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c15a5c0 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.610+0000 7f0c13666700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c15a8a0 con 0x7f0c0c1542f0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c0c0c9ad0 0x7f0c0c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c10c01700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0c0c1546d0 0x7f0c0c0c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:44956/0 (socket says 172.21.15.33:44956) 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c10c01700 1 -- 172.21.15.33:0/2333136866 learned_addr learned my addr 172.21.15.33:0/2333136866 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11402700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c1542f0 0x7f0c0c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0bfffff700 1 -- 172.21.15.33:0/2333136866 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391303509 0 0) 0x7f0c0c15b140 con 0x7f0c0c1546d0 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 -- 172.21.15.33:0/2333136866 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c1542f0 msgr2=0x7f0c0c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c1542f0 0x7f0c0c0cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 -- 172.21.15.33:0/2333136866 >> v1:172.21.15.42:6789/0 conn(0x7f0c0c1546d0 legacy=0x7f0c0c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 -- 172.21.15.33:0/2333136866 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c15ab80 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0c11c03700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0ce270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009d90 tx=0x7f0c0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2355ef3a30427c4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0bfffff700 1 -- 172.21.15.33:0/2333136866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400e070 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0bfffff700 1 -- 172.21.15.33:0/2333136866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0400bb20 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.611+0000 7f0bfffff700 1 -- 172.21.15.33:0/2333136866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04012430 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 -- 172.21.15.33:0/2333136866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0c9ad0 msgr2=0x7f0c0c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0ce270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009d90 tx=0x7f0c0400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 -- 172.21.15.33:0/2333136866 shutdown_connections 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0ce270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 --2- 172.21.15.33:0/2333136866 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c1542f0 0x7f0c0c0cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 -- 172.21.15.33:0/2333136866 >> 172.21.15.33:0/2333136866 conn(0x7f0c0c072230 msgr2=0x7f0c0c072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 -- 172.21.15.33:0/2333136866 shutdown_connections 2024-05-08T23:32:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.612+0000 7f0c13666700 1 -- 172.21.15.33:0/2333136866 wait complete. 2024-05-08T23:32:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.614+0000 7f0c13666700 1 Processor -- start 2024-05-08T23:32:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.614+0000 7f0c13666700 1 -- start start 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.614+0000 7f0c13666700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.614+0000 7f0c13666700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c0c1542f0 0x7f0c0c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c13666700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c13666700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c154ad0 con 0x7f0c0c1546d0 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c13666700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1f06a0 con 0x7f0c0c1542f0 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c13666700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c154d20 con 0x7f0c0c0c9ad0 2024-05-08T23:32:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56628/0 (socket says 172.21.15.33:56628) 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 -- 172.21.15.33:0/3823735416 learned_addr learned my addr 172.21.15.33:0/3823735416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c10c01700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c0c1542f0 0x7f0c0c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11402700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 -- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c0c9ad0 msgr2=0x7f0c0c0c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 -- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c0c1542f0 msgr2=0x7f0c0c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c0c1542f0 0x7f0c0c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 -- 172.21.15.33:0/3823735416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04009a70 con 0x7f0c0c1546d0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c11c03700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c04014040 tx=0x7f0c04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04012430 con 0x7f0c0c1546d0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04020070 con 0x7f0c0c1546d0 2024-05-08T23:32:25.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04004280 con 0x7f0c0c1546d0 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.615+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c155000 con 0x7f0c0c1546d0 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.616+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0c6970 con 0x7f0c0c1546d0 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.617+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c075460 con 0x7f0c0c1546d0 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.617+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0c040043e0 con 0x7f0c0c1546d0 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.618+0000 7f0bfdffb700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 0x7f0bf006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.618+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c04022070 con 0x7f0c0c1546d0 2024-05-08T23:32:25.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.618+0000 7f0c11402700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 0x7f0bf006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.618+0000 7f0c11402700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 0x7f0bf006d180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009be0 tx=0x7f0c00009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.622+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0c040560f0 con 0x7f0c0c1546d0 2024-05-08T23:32:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.839+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c0c0c6b30 con 0x7f0c0c1546d0 2024-05-08T23:32:25.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.841+0000 7f0bfdffb700 1 -- 172.21.15.33:0/3823735416 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f0c0402c090 con 0x7f0c0c1546d0 2024-05-08T23:32:25.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:25.841 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 msgr2=0x7f0bf006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 0x7f0bf006d180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009be0 tx=0x7f0c00009210 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 msgr2=0x7f0c0c0c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c04014040 tx=0x7f0c04004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 shutdown_connections 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0bf006acc0 0x7f0bf006d180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0c1546d0 0x7f0c0c0c6e90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0c0c0c9ad0 0x7f0c0c0c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 --2- 172.21.15.33:0/3823735416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c0c1542f0 0x7f0c0c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 >> 172.21.15.33:0/3823735416 conn(0x7f0c0c072230 msgr2=0x7f0c0c0c3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 shutdown_connections 2024-05-08T23:32:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.843+0000 7f0c13666700 1 -- 172.21.15.33:0/3823735416 wait complete. 2024-05-08T23:32:25.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.995+0000 7fb332e7f700 1 Processor -- start 2024-05-08T23:32:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.995+0000 7fb332e7f700 1 -- start start 2024-05-08T23:32:25.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb332e7f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c1522f0 0x7fb32c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb332e7f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb32c0b2de0 0x7fb32c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb332e7f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c1526d0 con 0x7fb32c0c77d0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb332e7f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c06bc20 con 0x7fb32c0b2de0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb332e7f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c1591e0 con 0x7fb32c1522f0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb32c0b2de0 0x7fb32c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb33141c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb32c0c77d0 0x7fb32c06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48546/0 (socket says 172.21.15.33:48546) 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb33141c700 1 -- 172.21.15.33:0/816812431 learned_addr learned my addr 172.21.15.33:0/816812431 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb330c1b700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c1522f0 0x7fb32c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 -- 172.21.15.33:0/816812431 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c1522f0 msgr2=0x7fb32c0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c1522f0 0x7fb32c0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 -- 172.21.15.33:0/816812431 >> v1:172.21.15.33:6789/0 conn(0x7fb32c0c77d0 legacy=0x7fb32c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 -- 172.21.15.33:0/816812431 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0c7e80 con 0x7fb32c0b2de0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.996+0000 7fb32bfff700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c0b2de0 0x7fb32c0c8cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb318009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02c6059222426a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb32b7fe700 1 -- 172.21.15.33:0/816812431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318011830 con 0x7fb32c0b2de0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb32b7fe700 1 -- 172.21.15.33:0/816812431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318011990 con 0x7fb32c0b2de0 2024-05-08T23:32:25.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb32b7fe700 1 -- 172.21.15.33:0/816812431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318011d20 con 0x7fb32c0b2de0 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 -- 172.21.15.33:0/816812431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c0b2de0 msgr2=0x7fb32c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c0b2de0 0x7fb32c0c8cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb318009b80 comp rx=0 tx=0).stop 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 -- 172.21.15.33:0/816812431 shutdown_connections 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c1522f0 0x7fb32c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 --2- 172.21.15.33:0/816812431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c0b2de0 0x7fb32c0c8cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 -- 172.21.15.33:0/816812431 >> 172.21.15.33:0/816812431 conn(0x7fb32c06ce60 msgr2=0x7fb32c06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 -- 172.21.15.33:0/816812431 shutdown_connections 2024-05-08T23:32:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.997+0000 7fb332e7f700 1 -- 172.21.15.33:0/816812431 wait complete. 2024-05-08T23:32:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.998+0000 7fb332e7f700 1 Processor -- start 2024-05-08T23:32:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 -- start start 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb32c0b2de0 0x7fb32c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c0c77d0 0x7fb32c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0cca80 con 0x7fb32c1522f0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c1e44e0 con 0x7fb32c0b2de0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:25.999+0000 7fb332e7f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0cc500 con 0x7fb32c0c77d0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56646/0 (socket says 172.21.15.33:56646) 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 -- 172.21.15.33:0/1261545078 learned_addr learned my addr 172.21.15.33:0/1261545078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb32bfff700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c0c77d0 0x7fb32c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb330c1b700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb32c0b2de0 0x7fb32c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 -- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c0c77d0 msgr2=0x7fb32c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c0c77d0 0x7fb32c1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 -- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb32c0b2de0 msgr2=0x7fb32c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb32c0b2de0 0x7fb32c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 -- 172.21.15.33:0/1261545078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180094a0 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb33141c700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb32400caf0 tx=0x7fb32400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400a9e0 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32400ab40 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c1e4200 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324012400 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.000+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c0cabd0 con 0x7fb32c1522f0 2024-05-08T23:32:26.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.001+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c073770 con 0x7fb32c1522f0 2024-05-08T23:32:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.002+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb32401b3b0 con 0x7fb32c1522f0 2024-05-08T23:32:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.002+0000 7fb3297fa700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 0x7fb31006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.003+0000 7fb330c1b700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 0x7fb31006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.003+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb324088d90 con 0x7fb32c1522f0 2024-05-08T23:32:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.003+0000 7fb330c1b700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 0x7fb31006d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b40 tx=0x7fb320009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:26.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.007+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb3240540b0 con 0x7fb32c1522f0 2024-05-08T23:32:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.277+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32c1560c0 con 0x7fb32c1522f0 2024-05-08T23:32:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.278+0000 7fb3297fa700 1 -- 172.21.15.33:0/1261545078 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7fb32c1560c0 con 0x7fb32c1522f0 2024-05-08T23:32:26.278 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:26.278 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 msgr2=0x7fb31006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 0x7fb31006d150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb320009b40 tx=0x7fb320009210 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 msgr2=0x7fb32c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb32400caf0 tx=0x7fb32400ce00 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 shutdown_connections 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb31006ac90 0x7fb31006d150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb32c1522f0 0x7fb32c1568e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb32c0b2de0 0x7fb32c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 --2- 172.21.15.33:0/1261545078 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb32c0c77d0 0x7fb32c1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 >> 172.21.15.33:0/1261545078 conn(0x7fb32c06ce60 msgr2=0x7fb32c14f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 shutdown_connections 2024-05-08T23:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:26.280+0000 7fb332e7f700 1 -- 172.21.15.33:0/1261545078 wait complete. 2024-05-08T23:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:32:26.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:32:10.330935+0000', 'modified': '2024-05-08T23:32:16.353975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.42:6833/391209758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 391209758}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 391209758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:32:26.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:32:26.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T23:32:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.048+0000 7fe1b9331700 1 Processor -- start 2024-05-08T23:32:27.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.048+0000 7fe1b9331700 1 -- start start 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b9331700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1b40c38a0 0x7fe1b40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b9331700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe1b40c9a60 0x7fe1b40c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b9331700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b4070450 con 0x7fe1b4152350 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b9331700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b40c2e90 con 0x7fe1b40c38a0 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b9331700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b4070730 con 0x7fe1b40c9a60 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b27fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe1b40c38a0 0x7fe1b40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe1b4152350 0x7fe1b4158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48560/0 (socket says 172.21.15.33:48560) 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.049+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2208346664 learned_addr learned my addr 172.21.15.33:0/2208346664 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/2208346664 >> v2:172.21.15.42:3300/0 conn(0x7fe1b40c9a60 0x7fe1b40c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b27fc700 1 -- 172.21.15.33:0/2208346664 >> v2:172.21.15.42:3300/0 conn(0x7fe1b40c9a60 msgr2=0x7fe1b40c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2208346664 >> v2:172.21.15.42:3300/0 conn(0x7fe1b40c9a60 0x7fe1b40c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b27fc700 1 -- 172.21.15.33:0/2208346664 >> v1:172.21.15.33:6789/0 conn(0x7fe1b4152350 legacy=0x7fe1b4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b27fc700 1 -- 172.21.15.33:0/2208346664 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4070a10 con 0x7fe1b40c38a0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2208346664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b40c38a0 0x7fe1b40c9520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406f590 tx=0x7fe1a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d6b949c8e323d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b1ffb700 1 -- 172.21.15.33:0/2208346664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8011830 con 0x7fe1b40c38a0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b1ffb700 1 -- 172.21.15.33:0/2208346664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8011990 con 0x7fe1b40c38a0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.050+0000 7fe1b1ffb700 1 -- 172.21.15.33:0/2208346664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8011d20 con 0x7fe1b40c38a0 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 -- 172.21.15.33:0/2208346664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b40c38a0 msgr2=0x7fe1b40c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 --2- 172.21.15.33:0/2208346664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b40c38a0 0x7fe1b40c9520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406f590 tx=0x7fe1a8009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 -- 172.21.15.33:0/2208346664 shutdown_connections 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 --2- 172.21.15.33:0/2208346664 >> v2:172.21.15.42:3300/0 conn(0x7fe1b40c9a60 0x7fe1b40c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 --2- 172.21.15.33:0/2208346664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b40c38a0 0x7fe1b40c9520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 -- 172.21.15.33:0/2208346664 >> 172.21.15.33:0/2208346664 conn(0x7fe1b40666c0 msgr2=0x7fe1b4066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 -- 172.21.15.33:0/2208346664 shutdown_connections 2024-05-08T23:32:27.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.051+0000 7fe1b9331700 1 -- 172.21.15.33:0/2208346664 wait complete. 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.052+0000 7fe1b9331700 1 Processor -- start 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.052+0000 7fe1b9331700 1 -- start start 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 0x7fe1b41510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 0x7fe1b4156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4154af0 con 0x7fe1b4152350 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4156680 con 0x7fe1b40c38a0 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b9331700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b406ca40 con 0x7fe1b40c9a60 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56664/0 (socket says 172.21.15.33:56664) 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b37fe700 1 -- 172.21.15.33:0/651183229 learned_addr learned my addr 172.21.15.33:0/651183229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 0x7fe1b41510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.053+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 0x7fe1b4156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 -- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 msgr2=0x7fe1b4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 0x7fe1b4156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 -- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 msgr2=0x7fe1b41510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 0x7fe1b41510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 -- 172.21.15.33:0/651183229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80094a0 con 0x7fe1b4152350 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 0x7fe1b4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 0x7fe1b41510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007e40 tx=0x7fe1ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe19b7fe700 1 -- 172.21.15.33:0/651183229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00eea0 con 0x7fe1b4152350 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe19b7fe700 1 -- 172.21.15.33:0/651183229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac014420 con 0x7fe1b4152350 2024-05-08T23:32:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b406cd00 con 0x7fe1b4152350 2024-05-08T23:32:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe19b7fe700 1 -- 172.21.15.33:0/651183229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0147b0 con 0x7fe1b4152350 2024-05-08T23:32:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.054+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b4154150 con 0x7fe1b4152350 2024-05-08T23:32:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.055+0000 7fe19b7fe700 1 -- 172.21.15.33:0/651183229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe1ac014910 con 0x7fe1b4152350 2024-05-08T23:32:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.056+0000 7fe19b7fe700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 0x7fe1a006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.056+0000 7fe19b7fe700 1 -- 172.21.15.33:0/651183229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1ac089a00 con 0x7fe1b4152350 2024-05-08T23:32:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.056+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 0x7fe1a006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.056+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 0x7fe1a006d180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005910 tx=0x7fe1a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.058+0000 7fe1c54ebb80 1 Processor -- start 2024-05-08T23:32:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.058+0000 7fe1c54ebb80 1 -- start start 2024-05-08T23:32:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 0x563ee2e3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e3d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2d54100 con 0x563ee2e26bc0 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2d51ac0 con 0x563ee2e26fa0 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee2d4d3b0 con 0x563ee2dff930 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56668/0 (socket says 172.21.15.33:56668) 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2637393295 learned_addr learned my addr 172.21.15.33:0/2637393295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e3d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.059+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 0x563ee2e3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 msgr2=0x563ee2e3cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 0x563ee2e3cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 msgr2=0x563ee2e3d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e3d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2637393295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ee2bce360 con 0x563ee2e26bc0 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 0x563ee2e3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005c20 tx=0x7fe1a4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dae18946ae964f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe198ff9700 1 -- 172.21.15.33:0/2637393295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4020450 con 0x563ee2e26bc0 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe198ff9700 1 -- 172.21.15.33:0/2637393295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a40205b0 con 0x563ee2e26bc0 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.060+0000 7fe198ff9700 1 -- 172.21.15.33:0/2637393295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4020920 con 0x563ee2e26bc0 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2637393295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 msgr2=0x563ee2e4bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005c20 tx=0x7fe1a4001e70 comp rx=0 tx=0).stop 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2637393295 shutdown_connections 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e3d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4bd20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2637393295 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2dff930 0x563ee2e3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2637393295 >> 172.21.15.33:0/2637393295 conn(0x563ee2d5b1c0 msgr2=0x563ee2dfb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2637393295 shutdown_connections 2024-05-08T23:32:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2637393295 wait complete. 2024-05-08T23:32:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.061+0000 7fe1c54ebb80 1 -- start start 2024-05-08T23:32:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 Processor -- start 2024-05-08T23:32:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e20a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2ce9510 con 0x563ee2e26bc0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2ce91b0 con 0x563ee2e26fa0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee2bd0650 con 0x563ee2e205a0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.062+0000 7fe1c54ebb80 10 client.0 _ll_register_callbacks cb 0x563ee2cf2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56674/0 (socket says 172.21.15.33:56674) 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2483950002 learned_addr learned my addr 172.21.15.33:0/2483950002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e20a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 msgr2=0x563ee2e20a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e20a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 msgr2=0x563ee2e56680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e56680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 -- 172.21.15.33:0/2483950002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b4070450 con 0x563ee2e26bc0 2024-05-08T23:32:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4008fd0 tx=0x7fe1a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc00976ae5aaf816 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4027070 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1a4022110 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ee2ca6220 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1a4020450 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1c54ebb80 10 client.4635 fetch_fsmap learned FSMap version 8 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.063+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563ee2b7ab10 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563ee2ca6220 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 4204+0+0 (secure 0 0 0) 0x7fe1a40222d0 con 0x563ee2e26bc0 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 20 client.4635 populate_metadata read hostname 'smithi033' 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 4 client.4635 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 --> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ee2dcc790 con 0x563ee2e50560 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe191ffb700 10 client.4635 ms_handle_connect on v2:172.21.15.42:6832/391209758 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2036445f94ad4672 server_cookie=af8362c1f786934d in_seq=0 out_seq=0 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 -- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 msgr2=0x563ee2e50940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 -- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 msgr2=0x563ee2e50940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.064+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 unknown :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.065+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 unknown :-1 s=BANNER_CONNECTING pgs=13 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.065+0000 7fe191ffb700 10 client.4635 ms_handle_connect on v2:172.21.15.42:6832/391209758 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.065+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 crc :-1 s=READY pgs=13 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2036445f94ad4672 server_cookie=af8362c1f786934d in_seq=0 out_seq=0 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe191ffb700 1 -- 172.21.15.33:0/2483950002 <== mds.0 v2:172.21.15.42:6832/391209758 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ee2dcc790 con 0x563ee2e50560 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe191ffb700 10 client.4635 handle_command_reply: tid=0 2024-05-08T23:32:27.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 client.4635 shutdown 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 msgr2=0x563ee2e4e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4008fd0 tx=0x7fe1a4010000 comp rx=0 tx=0).stop 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 shutdown_connections 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2e50560 0x563ee2e50940 crc :-1 s=READY pgs=13 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e20a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2e4e780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe191ffb700 0 client.4635 ms_handle_reset on v2:172.21.15.42:6832/391209758 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/2483950002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2e56680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 >> 172.21.15.33:0/2483950002 conn(0x563ee2d5b1c0 msgr2=0x563ee2dfbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 shutdown_connections 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/2483950002 wait complete. 2024-05-08T23:32:27.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.066+0000 7fe1c54ebb80 20 client.4635 trim_cache size 0 max 16384 2024-05-08T23:32:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.085+0000 7fe1c54ebb80 1 Processor -- start 2024-05-08T23:32:27.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.085+0000 7fe1c54ebb80 1 -- start start 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2df6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26bc0 0x563ee2df71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e257d0 con 0x563ee2e26fa0 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e1ff50 con 0x563ee2e26bc0 2024-05-08T23:32:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e255b0 con 0x563ee2e205a0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56682/0 (socket says 172.21.15.33:56682) 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1b37fe700 1 -- 172.21.15.33:0/3939328552 learned_addr learned my addr 172.21.15.33:0/3939328552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2df6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.086+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26bc0 0x563ee2df71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 -- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 msgr2=0x563ee2df6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2df6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 -- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26bc0 msgr2=0x563ee2df71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26bc0 0x563ee2df71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 -- 172.21.15.33:0/3939328552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ee2d13510 con 0x563ee2e26fa0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0588c0 tx=0x7fe1ac058500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76b916a5084f5be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1997fa700 1 -- 172.21.15.33:0/3939328552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac058930 con 0x563ee2e26fa0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1997fa700 1 -- 172.21.15.33:0/3939328552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac05c760 con 0x563ee2e26fa0 2024-05-08T23:32:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1997fa700 1 -- 172.21.15.33:0/3939328552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac060d20 con 0x563ee2e26fa0 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3939328552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 msgr2=0x563ee2e3f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0588c0 tx=0x7fe1ac058500 comp rx=0 tx=0).stop 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3939328552 shutdown_connections 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26fa0 0x563ee2e3f620 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2df6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.087+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3939328552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26bc0 0x563ee2df71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.088+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3939328552 >> 172.21.15.33:0/3939328552 conn(0x563ee2d5b1c0 msgr2=0x563ee2df4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.088+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3939328552 shutdown_connections 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.088+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3939328552 wait complete. 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.088+0000 7fe1c54ebb80 1 -- start start 2024-05-08T23:32:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.088+0000 7fe1c54ebb80 1 Processor -- start 2024-05-08T23:32:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e3cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2eaa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e3d790 con 0x563ee2e26bc0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e24d30 con 0x563ee2e26fa0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ee2e3d2c0 con 0x563ee2e205a0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1c54ebb80 10 client.0 _ll_register_callbacks cb 0x563ee2cf2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56686/0 (socket says 172.21.15.33:56686) 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 -- 172.21.15.33:0/3534291883 learned_addr learned my addr 172.21.15.33:0/3534291883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e3cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b37fe700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2eaa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 -- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 msgr2=0x563ee2e3cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e3cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 -- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 msgr2=0x563ee2eaa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2eaa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.089+0000 7fe1b27fc700 1 -- 172.21.15.33:0/3534291883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b4154150 con 0x563ee2e26bc0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1b27fc700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000c00 tx=0x7fe1a8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cc59c6cb083f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a801f040 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1a8004450 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1a40205b0 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1a80036a0 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 10 client.4641 fetch_fsmap learned FSMap version 8 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563ee2bd2c80 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a40205b0 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 8) v1 ==== 4204+0+0 (secure 0 0 0) 0x7fe1a8003800 con 0x563ee2e26bc0 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 10 client.4641 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 20 client.4641 populate_metadata read hostname 'smithi033' 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2df7950 0x563ee2df7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 4 client.4641 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:32:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.090+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 --> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ee2d45e70 con 0x563ee2df7950 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2df7950 0x563ee2df7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe191ffb700 10 client.4641 ms_handle_connect on v2:172.21.15.42:6832/391209758 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1b2ffd700 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2df7950 0x563ee2df7d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57926f373f6a439 server_cookie=2ce426c3b9c9f05b in_seq=0 out_seq=0 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe191ffb700 1 -- 172.21.15.33:0/3534291883 <== mds.0 v2:172.21.15.42:6832/391209758 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x563ee2d45e70 con 0x563ee2df7950 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe191ffb700 10 client.4641 handle_command_reply: tid=0 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 client.4641 shutdown 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 msgr2=0x563ee2eb3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000c00 tx=0x7fe1a8010000 comp rx=0 tx=0).stop 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 shutdown_connections 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:6832/391209758,v1:172.21.15.42:6833/391209758] conn(0x563ee2df7950 0x563ee2df7d30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee2e26fa0 0x563ee2eaa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x563ee2e205a0 0x563ee2e3cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe191ffb700 0 client.4641 ms_handle_reset on v2:172.21.15.42:6832/391209758 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 --2- 172.21.15.33:0/3534291883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ee2e26bc0 0x563ee2eb3fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 >> 172.21.15.33:0/3534291883 conn(0x563ee2d5b1c0 msgr2=0x563ee2e3f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 shutdown_connections 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.091+0000 7fe1c54ebb80 1 -- 172.21.15.33:0/3534291883 wait complete. 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1c54ebb80 20 client.4641 trim_cache size 0 max 16384 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4281, 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T23:32:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 16.759918132999999, 2024-05-08T23:32:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 8, 2024-05-08T23:32:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 21, 2024-05-08T23:32:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 21, 2024-05-08T23:32:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 23.384660967999999 2024-05-08T23:32:27.100 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 msgr2=0x7fe1a006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 0x7fe1a006d180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4005910 tx=0x7fe1a4009040 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 msgr2=0x7fe1b41467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007e40 tx=0x7fe1ac00e3c0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 shutdown_connections 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe1a006acc0 0x7fe1a006d180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4152350 0x7fe1b41467d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b40c38a0 0x7fe1b41510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 --2- 172.21.15.33:0/651183229 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe1b40c9a60 0x7fe1b4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 >> 172.21.15.33:0/651183229 conn(0x7fe1b40666c0 msgr2=0x7fe1b40cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 shutdown_connections 2024-05-08T23:32:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.092+0000 7fe1b9331700 1 -- 172.21.15.33:0/651183229 wait complete. 2024-05-08T23:32:27.106 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:32:27.116 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:32:27.117 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:32:27.117 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:32:27.117 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T23:32:27.175 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T23:32:27.175 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T23:32:27.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.175 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T23:32:27.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-08T23:32:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-05-08T23:32:27.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:32:27.276 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-08T23:32:27.276 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:32:27.276 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:27.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T23:32:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.463+0000 7fb544c2c700 1 Processor -- start 2024-05-08T23:32:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.463+0000 7fb544c2c700 1 -- start start 2024-05-08T23:32:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb544c2c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8a50 0x7fb540158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb544c2c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5400c38a0 0x7fb540151e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb544c2c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400cdd30 con 0x7fb5400c8e30 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb544c2c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb540070410 con 0x7fb5400c38a0 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb544c2c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb540158610 con 0x7fb5400c8a50 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb53dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5400c38a0 0x7fb540151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb53dd9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb5400c38a0 0x7fb540151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56696/0 (socket says 172.21.15.33:56696) 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.464+0000 7fb53dd9b700 1 -- 172.21.15.33:0/832523854 learned_addr learned my addr 172.21.15.33:0/832523854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53e59c700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8a50 0x7fb540158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53dd9b700 1 -- 172.21.15.33:0/832523854 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8a50 msgr2=0x7fb540158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53dd9b700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8a50 0x7fb540158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53dd9b700 1 -- 172.21.15.33:0/832523854 >> v1:172.21.15.42:6789/0 conn(0x7fb5400c8e30 legacy=0x7fb54015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53dd9b700 1 -- 172.21.15.33:0/832523854 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540158fe0 con 0x7fb5400c38a0 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53dd9b700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c38a0 0x7fb540151e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5280098f0 tx=0x7fb528009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1070d9e600200100 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53d59a700 1 -- 172.21.15.33:0/832523854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800b8a0 con 0x7fb5400c38a0 2024-05-08T23:32:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53d59a700 1 -- 172.21.15.33:0/832523854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800ba00 con 0x7fb5400c38a0 2024-05-08T23:32:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.465+0000 7fb53d59a700 1 -- 172.21.15.33:0/832523854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800bd90 con 0x7fb5400c38a0 2024-05-08T23:32:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 -- 172.21.15.33:0/832523854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c38a0 msgr2=0x7fb540151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c38a0 0x7fb540151e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5280098f0 tx=0x7fb528009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 -- 172.21.15.33:0/832523854 shutdown_connections 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8a50 0x7fb540158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 --2- 172.21.15.33:0/832523854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c38a0 0x7fb540151e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 -- 172.21.15.33:0/832523854 >> 172.21.15.33:0/832523854 conn(0x7fb540072230 msgr2=0x7fb540072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 -- 172.21.15.33:0/832523854 shutdown_connections 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.466+0000 7fb544c2c700 1 -- 172.21.15.33:0/832523854 wait complete. 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.467+0000 7fb544c2c700 1 Processor -- start 2024-05-08T23:32:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.467+0000 7fb544c2c700 1 -- start start 2024-05-08T23:32:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5400c38a0 0x7fb540150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8e30 0x7fb540145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540148f20 con 0x7fb5400c8a50 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540148c60 con 0x7fb5400c38a0 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb544c2c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5401f1910 con 0x7fb5400c8e30 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56710/0 (socket says 172.21.15.33:56710) 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 -- 172.21.15.33:0/292643831 learned_addr learned my addr 172.21.15.33:0/292643831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53e59c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5400c38a0 0x7fb540150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53ed9d700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8e30 0x7fb540145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 -- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8e30 msgr2=0x7fb540145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8e30 0x7fb540145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 -- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5400c38a0 msgr2=0x7fb540150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5400c38a0 0x7fb540150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.468+0000 7fb53dd9b700 1 -- 172.21.15.33:0/292643831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528009550 con 0x7fb5400c8a50 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb53dd9b700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb528000c00 tx=0x7fb528004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528004380 con 0x7fb5400c8a50 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52801f070 con 0x7fb5400c8a50 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540150850 con 0x7fb5400c8a50 2024-05-08T23:32:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5280036a0 con 0x7fb5400c8a50 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.469+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540147aa0 con 0x7fb5400c8a50 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.470+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb540075460 con 0x7fb5400c8a50 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.471+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb528021030 con 0x7fb5400c8a50 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.471+0000 7fb536ffd700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 0x7fb52006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.471+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb52808a690 con 0x7fb5400c8a50 2024-05-08T23:32:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.471+0000 7fb53e59c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 0x7fb52006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.472+0000 7fb53e59c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 0x7fb52006d150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ab60 tx=0x7fb530008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.476+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb528055880 con 0x7fb5400c8a50 2024-05-08T23:32:27.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.743+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5401f1a50 con 0x7fb5400c8a50 2024-05-08T23:32:27.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.743+0000 7fb536ffd700 1 -- 172.21.15.33:0/292643831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb528058ea0 con 0x7fb5400c8a50 2024-05-08T23:32:27.744 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 msgr2=0x7fb52006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 0x7fb52006d150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb53000ab60 tx=0x7fb530008040 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 msgr2=0x7fb54014ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb528000c00 tx=0x7fb528004750 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 shutdown_connections 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb52006ac90 0x7fb52006d150 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb5400c8e30 0x7fb540145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5400c38a0 0x7fb540150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 --2- 172.21.15.33:0/292643831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5400c8a50 0x7fb54014ad80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 >> 172.21.15.33:0/292643831 conn(0x7fb540072230 msgr2=0x7fb54014c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.745+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 shutdown_connections 2024-05-08T23:32:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:27.746+0000 7fb544c2c700 1 -- 172.21.15.33:0/292643831 wait complete. 2024-05-08T23:32:27.759 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:32:27.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:32:27.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-08T23:32:27.759 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:32:27.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:27.760 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:32:27.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:32:27.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:32:27.760 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:27.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.760 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-08T23:32:27.779 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:32:27.779 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:32:27.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:32:27.779 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:27.779 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42208sec preferred_lft 42208sec 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-05-08T23:32:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:32:27.781 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:32:27.781 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:32:27.781 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-08T23:32:27.781 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:32:27.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.781 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T23:32:27.781 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T23:32:27.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:32:27.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-08T23:32:27.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-08T23:32:27.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:32:27.782 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T23:32:27.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.962 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:32:27.995 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-08T23:32:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:27.997 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-08T23:32:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-08T23:32:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:32:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:32:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:32:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:32:28.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.077 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T23:32:28.077 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T23:32:28.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:32:28.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:32:28.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:32:28.078 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T23:32:28.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.322 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T23:32:28.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.345 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-08T23:32:28.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.406 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T23:32:28.406 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T23:32:28.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:32:28.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:32:28.406 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T23:32:28.518 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:32:28.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:32:28.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:32:28.519 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T23:32:28.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.737 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T23:32:28.737 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T23:32:28.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-08T23:32:28.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:32:28.737 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T23:32:28.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:28.850 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:28.869 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:32:28.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:32:28.870 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:28.930 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T23:32:28.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:28.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:32:29.029 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:32:29.030 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.42:3300,172.21.15.42:3301" 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with new device syntax: 0@7b439b97-64c0-4aab-ac3b-1b32ed376bfb.cephfs=/ 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.33:3300/172.21.15.42:3300/172.21.15.42:3301" will pass to kernel 2024-05-08T23:32:29.122 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:32:29.123 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:32:29.123 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.42:3300,172.21.15.42:3301" 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with new device syntax: 0@7b439b97-64c0-4aab-ac3b-1b32ed376bfb.cephfs=/ 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.33:3300/172.21.15.42:3300/172.21.15.42:3301" will pass to kernel 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: v2 mount support check returned 2 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: kernel does not support v2 syntax 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:6789,172.21.15.42:6789,172.21.15.42:6790" 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:6789,172.21.15.42:6789,172.21.15.42:6790:/ 2024-05-08T23:32:29.142 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=7b439b97-64c0-4aab-ac3b-1b32ed376bfb" will pass to kernel 2024-05-08T23:32:29.145 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T23:32:29.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:29.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:29.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:29.177 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:32:29.603 INFO:teuthology.orchestra.run.smithi033.stdout:client4499 2024-05-08T23:32:29.604 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:32:29.604 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:29.626 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:32:29.732 INFO:teuthology.orchestra.run.smithi033.stdout:client4499 2024-05-08T23:32:29.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:29.732 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T23:32:29.793 INFO:teuthology.orchestra.run.smithi033.stdout:client4499 2024-05-08T23:32:29.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:29.793 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/7b439b97-64c0-4aab-ac3b-1b32ed376bfb.client4499/status') 2024-05-08T23:32:30.129 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.4499 (3)172.21.15.33:0/1854248010 2024-05-08T23:32:30.129 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-05-08T23:32:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-05-08T23:32:30.129 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-05-08T23:32:30.130 INFO:teuthology.orchestra.run.smithi033.stderr:70 bytes copied, 2.9532e-05 s, 2.4 MB/s 2024-05-08T23:32:30.131 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-05-08T23:32:30.142 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-05-08T23:32:30.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.286+0000 7f1499a7b700 1 Processor -- start 2024-05-08T23:32:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.286+0000 7f1499a7b700 1 -- start start 2024-05-08T23:32:30.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.287+0000 7f1499a7b700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 0x7f14940c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.287+0000 7f1499a7b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f149406d980 0x7f149415e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.287+0000 7f1499a7b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14940c2860 con 0x7f14940712f0 2024-05-08T23:32:30.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.287+0000 7f1499a7b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940c3540 con 0x7f149406d980 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.287+0000 7f1499a7b700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940c9600 con 0x7f1494154540 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1492ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f14940712f0 0x7f149415d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48562/0 (socket says 172.21.15.33:48562) 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f149406d980 0x7f149415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1492ffd700 1 -- 172.21.15.33:0/1984124400 learned_addr learned my addr 172.21.15.33:0/1984124400 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14937fe700 1 --2- 172.21.15.33:0/1984124400 >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 0x7f14940c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 -- 172.21.15.33:0/1984124400 >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 msgr2=0x7f14940c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 --2- 172.21.15.33:0/1984124400 >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 0x7f14940c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 -- 172.21.15.33:0/1984124400 >> v1:172.21.15.33:6789/0 conn(0x7f14940712f0 legacy=0x7f149415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 -- 172.21.15.33:0/1984124400 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940c8620 con 0x7f149406d980 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14927fc700 1 -- 172.21.15.33:0/1984124400 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65212405 0 0) 0x7f14940c2860 con 0x7f14940712f0 2024-05-08T23:32:30.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14937fe700 1 --2- 172.21.15.33:0/1984124400 >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 0x7f14940c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f1493fff700 1 --2- 172.21.15.33:0/1984124400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149406d980 0x7f149415e000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f149404f2b0 tx=0x7f148c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a54859a3e64b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14927fc700 1 -- 172.21.15.33:0/1984124400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00e070 con 0x7f149406d980 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14927fc700 1 -- 172.21.15.33:0/1984124400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00bbd0 con 0x7f149406d980 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.288+0000 7f14927fc700 1 -- 172.21.15.33:0/1984124400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c012430 con 0x7f149406d980 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 -- 172.21.15.33:0/1984124400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149406d980 msgr2=0x7f149415e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 --2- 172.21.15.33:0/1984124400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149406d980 0x7f149415e000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f149404f2b0 tx=0x7f148c00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 -- 172.21.15.33:0/1984124400 shutdown_connections 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 --2- 172.21.15.33:0/1984124400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f149406d980 0x7f149415e000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 --2- 172.21.15.33:0/1984124400 >> v2:172.21.15.42:3300/0 conn(0x7f1494154540 0x7f14940c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 -- 172.21.15.33:0/1984124400 >> 172.21.15.33:0/1984124400 conn(0x7f149406ce60 msgr2=0x7f149406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 -- 172.21.15.33:0/1984124400 shutdown_connections 2024-05-08T23:32:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.289+0000 7f1499a7b700 1 -- 172.21.15.33:0/1984124400 wait complete. 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.291+0000 7f1499a7b700 1 Processor -- start 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.291+0000 7f1499a7b700 1 -- start start 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149406d980 0x7f14940cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1494154540 0x7f14941f1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149406c390 con 0x7f14940712f0 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149415b900 con 0x7f149406d980 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1499a7b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149415b660 con 0x7f1494154540 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56730/0 (socket says 172.21.15.33:56730) 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1492ffd700 1 -- 172.21.15.33:0/3467832648 learned_addr learned my addr 172.21.15.33:0/3467832648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f14937fe700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149406d980 0x7f14940cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.292+0000 7f1493fff700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1494154540 0x7f14941f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 -- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1494154540 msgr2=0x7f14941f1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1494154540 0x7f14941f1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 -- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149406d980 msgr2=0x7f14940cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149406d980 0x7f14940cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 -- 172.21.15.33:0/3467832648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009a70 con 0x7f14940712f0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1492ffd700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14880011e0 tx=0x7f148800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488004c80 con 0x7f14940712f0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488004e00 con 0x7f14940712f0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149406b5d0 con 0x7f14940712f0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488007310 con 0x7f14940712f0 2024-05-08T23:32:30.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.293+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149406bb90 con 0x7f14940712f0 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.294+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1494073770 con 0x7f14940712f0 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.295+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1488007470 con 0x7f14940712f0 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.295+0000 7f147bfff700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 0x7f148006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.295+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1488088fd0 con 0x7f14940712f0 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.295+0000 7f14937fe700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 0x7f148006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.296+0000 7f14937fe700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 0x7f148006d150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f148400c940 tx=0x7f1484008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.300+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f14880541b0 con 0x7f14940712f0 2024-05-08T23:32:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.518+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1494075460 con 0x7f14940712f0 2024-05-08T23:32:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.519+0000 7f147bfff700 1 -- 172.21.15.33:0/3467832648 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f14880577d0 con 0x7f14940712f0 2024-05-08T23:32:30.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:30.519 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 msgr2=0x7f148006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 0x7f148006d150 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f148400c940 tx=0x7f1484008040 comp rx=0 tx=0).stop 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 msgr2=0x7f14941f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14880011e0 tx=0x7f148800ca30 comp rx=0 tx=0).stop 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 shutdown_connections 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f148006ac90 0x7f148006d150 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1494154540 0x7f14941f1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f149406d980 0x7f14940cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 --2- 172.21.15.33:0/3467832648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14940712f0 0x7f14941f0de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.521+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 >> 172.21.15.33:0/3467832648 conn(0x7f149406ce60 msgr2=0x7f1494150990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.522+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 shutdown_connections 2024-05-08T23:32:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.522+0000 7f1499a7b700 1 -- 172.21.15.33:0/3467832648 wait complete. 2024-05-08T23:32:30.536 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:08.545052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:09.301342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:30.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.676+0000 7fb425592700 1 Processor -- start 2024-05-08T23:32:30.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.676+0000 7fb425592700 1 -- start start 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb425592700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb420154700 0x7fb4200ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb425592700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4200cc340 con 0x7fb42015dc50 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb425592700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42015a4b0 con 0x7fb42006d350 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb425592700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb42015a2a0 con 0x7fb420154700 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb41f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb42015dc50 0x7fb42015e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48564/0 (socket says 172.21.15.33:48564) 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb41f7fe700 1 -- 172.21.15.33:0/3934244091 learned_addr learned my addr 172.21.15.33:0/3934244091 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb41effd700 1 --2- 172.21.15.33:0/3934244091 >> v2:172.21.15.42:3300/0 conn(0x7fb420154700 0x7fb4200ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15103842 0 0) 0x7fb4200cc340 con 0x7fb42015dc50 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.677+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c0035b0 con 0x7fb42015dc50 2024-05-08T23:32:30.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485899632 0 0) 0x7fb40c0035b0 con 0x7fb42015dc50 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 >> v2:172.21.15.42:3300/0 conn(0x7fb420154700 msgr2=0x7fb4200ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 --2- 172.21.15.33:0/3934244091 >> v2:172.21.15.42:3300/0 conn(0x7fb420154700 0x7fb4200ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 >> v1:172.21.15.42:6789/0 conn(0x7fb42006d350 legacy=0x7fb4200cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42015a790 con 0x7fb42015dc50 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fb4180019f0 con 0x7fb42015dc50 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 >> v1:172.21.15.33:6789/0 conn(0x7fb42015dc50 legacy=0x7fb42015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 0x7fb40c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb40c004400 0x7fb40c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c0050d0 0x7fb40c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42015a2a0 con 0x7fb40c003a00 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c00c030 con 0x7fb40c004400 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c00c1f0 con 0x7fb40c0050d0 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 0x7fb40c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41e7fc700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb40c004400 0x7fb40c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41f7fe700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c0050d0 0x7fb40c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 -- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c0050d0 msgr2=0x7fb40c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c0050d0 0x7fb40c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 -- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb40c004400 msgr2=0x7fb40c0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb40c004400 0x7fb40c0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 -- 172.21.15.33:0/3934244091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0035b0 con 0x7fb40c003a00 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41f7fe700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c0050d0 0x7fb40c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41f7fe700 1 -- 172.21.15.33:0/3934244091 reap_dead start 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41effd700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 0x7fb40c003de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb410009b60 tx=0x7fb40c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f0785ddb1b9ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000e6c0 con 0x7fb40c003a00 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41000e820 con 0x7fb40c003a00 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.678+0000 7fb41dffb700 1 -- 172.21.15.33:0/3934244091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41000eb90 con 0x7fb40c003a00 2024-05-08T23:32:30.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 -- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 msgr2=0x7fb40c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 0x7fb40c003de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb410009b60 tx=0x7fb40c005880 comp rx=0 tx=0).stop 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 -- 172.21.15.33:0/3934244091 shutdown_connections 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 --2- 172.21.15.33:0/3934244091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c003a00 0x7fb40c003de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 -- 172.21.15.33:0/3934244091 >> 172.21.15.33:0/3934244091 conn(0x7fb420066cf0 msgr2=0x7fb42006ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 -- 172.21.15.33:0/3934244091 shutdown_connections 2024-05-08T23:32:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.679+0000 7fb425592700 1 -- 172.21.15.33:0/3934244091 wait complete. 2024-05-08T23:32:30.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 Processor -- start 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 -- start start 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c003a00 0x7fb4200c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4201f2b40 0x7fb4200c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4200c9390 con 0x7fb4200d07f0 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42015b5c0 con 0x7fb4201f2b40 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.681+0000 7fb425592700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42015b860 con 0x7fb40c003a00 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56744/0 (socket says 172.21.15.33:56744) 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 -- 172.21.15.33:0/1422282887 learned_addr learned my addr 172.21.15.33:0/1422282887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41f7fe700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4201f2b40 0x7fb4200c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41effd700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c003a00 0x7fb4200c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 -- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c003a00 msgr2=0x7fb4200c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c003a00 0x7fb4200c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 -- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4201f2b40 msgr2=0x7fb4200c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4201f2b40 0x7fb4200c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 -- 172.21.15.33:0/1422282887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42015bb40 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb41e7fc700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb40800f830 tx=0x7fb40800fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080052e0 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408005440 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42015be20 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080038c0 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.682+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42015c3e0 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.684+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb420075460 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.684+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb408003a20 con 0x7fb4200d07f0 2024-05-08T23:32:30.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.685+0000 7fb4177fe700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 0x7fb40006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:30.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.685+0000 7fb41effd700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 0x7fb40006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:30.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.685+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4080890d0 con 0x7fb4200d07f0 2024-05-08T23:32:30.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.685+0000 7fb41effd700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 0x7fb40006d150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4100090b0 tx=0x7fb41001e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:30.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.689+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fb4080542b0 con 0x7fb4200d07f0 2024-05-08T23:32:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.904+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4200ca000 con 0x7fb4200d07f0 2024-05-08T23:32:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.905+0000 7fb4177fe700 1 -- 172.21.15.33:0/1422282887 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4080578d0 con 0x7fb4200d07f0 2024-05-08T23:32:30.905 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:32:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 msgr2=0x7fb40006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 0x7fb40006d150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb4100090b0 tx=0x7fb41001e040 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 msgr2=0x7fb4200b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb40800f830 tx=0x7fb40800fb40 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 shutdown_connections 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb40006ac90 0x7fb40006d150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4201f2b40 0x7fb4200c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb40c003a00 0x7fb4200c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 --2- 172.21.15.33:0/1422282887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4200d07f0 0x7fb4200b47f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 >> 172.21.15.33:0/1422282887 conn(0x7fb420066cf0 msgr2=0x7fb42006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 shutdown_connections 2024-05-08T23:32:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:30.907+0000 7fb425592700 1 -- 172.21.15.33:0/1422282887 wait complete. 2024-05-08T23:32:30.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.060+0000 7f6ff7f79700 1 Processor -- start 2024-05-08T23:32:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.060+0000 7f6ff7f79700 1 -- start start 2024-05-08T23:32:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ff00c9580 0x7f6ff00c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 0x7f6ff0072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6ff0071590 0x7f6ff0071a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff015b4a0 con 0x7f6ff00c9580 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff015a5b0 con 0x7f6ff0071590 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff7f79700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0071f60 con 0x7f6ff00c7890 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ff00c9580 0x7f6ff00c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ff00c9580 0x7f6ff00c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56752/0 (socket says 172.21.15.33:56752) 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 -- 172.21.15.33:0/3147743679 learned_addr learned my addr 172.21.15.33:0/3147743679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff6516700 1 --2- 172.21.15.33:0/3147743679 >> v2:172.21.15.42:3300/0 conn(0x7f6ff0071590 0x7f6ff0071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5514700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 0x7f6ff0072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 -- 172.21.15.33:0/3147743679 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 msgr2=0x7f6ff0072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 0x7f6ff0072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 -- 172.21.15.33:0/3147743679 >> v2:172.21.15.42:3300/0 conn(0x7f6ff0071590 msgr2=0x7f6ff0071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/3147743679 >> v2:172.21.15.42:3300/0 conn(0x7f6ff0071590 0x7f6ff0071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 -- 172.21.15.33:0/3147743679 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff015a820 con 0x7f6ff00c9580 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5514700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 0x7f6ff0072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.061+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c9580 0x7f6ff00c9960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ff006fce0 tx=0x7f6fec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cfcc369b9297d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff4d13700 1 -- 172.21.15.33:0/3147743679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011830 con 0x7f6ff00c9580 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff4d13700 1 -- 172.21.15.33:0/3147743679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec011990 con 0x7f6ff00c9580 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff4d13700 1 -- 172.21.15.33:0/3147743679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011d20 con 0x7f6ff00c9580 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 -- 172.21.15.33:0/3147743679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c9580 msgr2=0x7f6ff00c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c9580 0x7f6ff00c9960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6ff006fce0 tx=0x7f6fec009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 -- 172.21.15.33:0/3147743679 shutdown_connections 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c9580 0x7f6ff00c9960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/3147743679 >> v2:172.21.15.42:3300/0 conn(0x7f6ff0071590 0x7f6ff0071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/3147743679 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c7890 0x7f6ff0072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 -- 172.21.15.33:0/3147743679 >> 172.21.15.33:0/3147743679 conn(0x7f6ff006eea0 msgr2=0x7f6ff006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 -- 172.21.15.33:0/3147743679 shutdown_connections 2024-05-08T23:32:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.062+0000 7f6ff7f79700 1 -- 172.21.15.33:0/3147743679 wait complete. 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 Processor -- start 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 -- start start 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 0x7f6ff0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c9580 0x7f6ff01f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0155400 con 0x7f6ff00c7890 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff01f3d40 con 0x7f6ff0071590 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.064+0000 7f6ff7f79700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff01547f0 con 0x7f6ff00c9580 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56758/0 (socket says 172.21.15.33:56758) 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 -- 172.21.15.33:0/4174220054 learned_addr learned my addr 172.21.15.33:0/4174220054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 0x7f6ff0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff6516700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c9580 0x7f6ff01f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 -- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c9580 msgr2=0x7f6ff01f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c9580 0x7f6ff01f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 -- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 msgr2=0x7f6ff0156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 0x7f6ff0156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 -- 172.21.15.33:0/4174220054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec009550 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 0x7f6ff0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff5514700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e810 tx=0x7f6fe000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000c020 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0004250 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff0154ad0 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00036a0 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.065+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff0155090 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.066+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff0155570 con 0x7f6ff00c7890 2024-05-08T23:32:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.067+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6fe0003800 con 0x7f6ff00c7890 2024-05-08T23:32:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.067+0000 7f6fe67fc700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 0x7f6fd406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.067+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fe0089160 con 0x7f6ff00c7890 2024-05-08T23:32:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.067+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 0x7f6fd406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.068+0000 7f6ff5d15700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 0x7f6fd406d150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fec014040 tx=0x7f6fec008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.072+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6fe00542b0 con 0x7f6ff00c7890 2024-05-08T23:32:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.290+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ff0155790 con 0x7f6ff00c7890 2024-05-08T23:32:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.290+0000 7f6fe67fc700 1 -- 172.21.15.33:0/4174220054 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fe00578d0 con 0x7f6ff00c7890 2024-05-08T23:32:31.291 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T23:32:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 msgr2=0x7f6fd406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 0x7f6fd406d150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fec014040 tx=0x7f6fec008030 comp rx=0 tx=0).stop 2024-05-08T23:32:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 msgr2=0x7f6ff0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000e810 tx=0x7f6fe000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 shutdown_connections 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6fd406ac90 0x7f6fd406d150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ff00c9580 0x7f6ff01f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff0071590 0x7f6ff0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 --2- 172.21.15.33:0/4174220054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ff00c7890 0x7f6ff0156e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 >> 172.21.15.33:0/4174220054 conn(0x7f6ff006eea0 msgr2=0x7f6ff00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 shutdown_connections 2024-05-08T23:32:31.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.293+0000 7f6ff7f79700 1 -- 172.21.15.33:0/4174220054 wait complete. 2024-05-08T23:32:31.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.446+0000 7fe118bf8700 1 Processor -- start 2024-05-08T23:32:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.446+0000 7fe118bf8700 1 -- start start 2024-05-08T23:32:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.447+0000 7fe118bf8700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe11415e3d0 0x7fe1140ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.447+0000 7fe118bf8700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11406e1e0 con 0x7fe1140b30b0 2024-05-08T23:32:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.447+0000 7fe118bf8700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11406e720 con 0x7fe114154480 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.447+0000 7fe118bf8700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11406e440 con 0x7fe11415e3d0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe111d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe1140b30b0 0x7fe11406d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48578/0 (socket says 172.21.15.33:48578) 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe111d9b700 1 -- 172.21.15.33:0/3967676340 learned_addr learned my addr 172.21.15.33:0/3967676340 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe112d9d700 1 --2- 172.21.15.33:0/3967676340 >> v2:172.21.15.42:3300/0 conn(0x7fe11415e3d0 0x7fe1140ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002866804 0 0) 0x7fe11406e1e0 con 0x7fe1140b30b0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1000035b0 con 0x7fe1140b30b0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827477453 0 0) 0x7fe11406e720 con 0x7fe114154480 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11406e1e0 con 0x7fe114154480 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3014176911 0 0) 0x7fe1000035b0 con 0x7fe1140b30b0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 >> v2:172.21.15.42:3300/0 conn(0x7fe11415e3d0 msgr2=0x7fe1140ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 --2- 172.21.15.33:0/3967676340 >> v2:172.21.15.42:3300/0 conn(0x7fe11415e3d0 0x7fe1140ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 >> v1:172.21.15.42:6789/0 conn(0x7fe114154480 legacy=0x7fe11406eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11415d8f0 con 0x7fe1140b30b0 2024-05-08T23:32:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fe0fc0015f0 con 0x7fe1140b30b0 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 >> v1:172.21.15.33:6789/0 conn(0x7fe1140b30b0 legacy=0x7fe11406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100003be0 0x7fe100003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100004850 0x7fe100004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 0x7fe10000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11406e440 con 0x7fe100005210 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10000c030 con 0x7fe100003be0 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.448+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10000c1f0 con 0x7fe100004850 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 0x7fe10000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe11259c700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100003be0 0x7fe100003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe111d9b700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100004850 0x7fe100004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 -- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100004850 msgr2=0x7fe100004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100004850 0x7fe100004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 -- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100003be0 msgr2=0x7fe100003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100003be0 0x7fe100003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 -- 172.21.15.33:0/3967676340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000035b0 con 0x7fe100005210 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe11259c700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100003be0 0x7fe100003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 -- 172.21.15.33:0/3967676340 reap_dead start 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe112d9d700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 0x7fe10000ba10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fc0 tx=0x7fe100005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5eb380873b481f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003a20 con 0x7fe100005210 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00f040 con 0x7fe100005210 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.449+0000 7fe11159a700 1 -- 172.21.15.33:0/3967676340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003da0 con 0x7fe100005210 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 -- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 msgr2=0x7fe10000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 0x7fe10000ba10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fc0 tx=0x7fe100005880 comp rx=0 tx=0).stop 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 -- 172.21.15.33:0/3967676340 shutdown_connections 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 --2- 172.21.15.33:0/3967676340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe100005210 0x7fe10000ba10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 -- 172.21.15.33:0/3967676340 >> 172.21.15.33:0/3967676340 conn(0x7fe114072670 msgr2=0x7fe114072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 -- 172.21.15.33:0/3967676340 shutdown_connections 2024-05-08T23:32:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.450+0000 7fe118bf8700 1 -- 172.21.15.33:0/3967676340 wait complete. 2024-05-08T23:32:31.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.451+0000 7fe118bf8700 1 Processor -- start 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 -- start start 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100005210 0x7fe114150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe11414ff40 0x7fe11415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140ca700 con 0x7fe114151f00 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11414f3a0 con 0x7fe11414ff40 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe118bf8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141528d0 con 0x7fe100005210 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe112d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe112d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56770/0 (socket says 172.21.15.33:56770) 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.452+0000 7fe112d9d700 1 -- 172.21.15.33:0/2326048739 learned_addr learned my addr 172.21.15.33:0/2326048739 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe111d9b700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe11414ff40 0x7fe11415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe11259c700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100005210 0x7fe114150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 -- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100005210 msgr2=0x7fe114150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100005210 0x7fe114150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 -- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe11414ff40 msgr2=0x7fe11415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe11414ff40 0x7fe11415c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 -- 172.21.15.33:0/2326048739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1141535b0 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe112d9d700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e700 tx=0x7fe10c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003e10 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c004250 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe114150ab0 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c004600 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.453+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1140ca0b0 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.454+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140c50e0 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.455+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe10c004760 con 0x7fe114151f00 2024-05-08T23:32:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.455+0000 7fe10affd700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 0x7fe0f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.455+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe10c08a2d0 con 0x7fe114151f00 2024-05-08T23:32:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.455+0000 7fe11259c700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 0x7fe0f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.456+0000 7fe11259c700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 0x7fe0f406d150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe104000c00 tx=0x7fe104009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.460+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe10c055540 con 0x7fe114151f00 2024-05-08T23:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.682+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe114151c00 con 0x7fe114151f00 2024-05-08T23:32:31.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.682+0000 7fe10affd700 1 -- 172.21.15.33:0/2326048739 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe10c058b60 con 0x7fe114151f00 2024-05-08T23:32:31.683 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T23:32:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 msgr2=0x7fe0f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 0x7fe0f406d150 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe104000c00 tx=0x7fe104009040 comp rx=0 tx=0).stop 2024-05-08T23:32:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 msgr2=0x7fe114152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e700 tx=0x7fe10c009750 comp rx=0 tx=0).stop 2024-05-08T23:32:31.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 shutdown_connections 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe0f406ac90 0x7fe0f406d150 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114151f00 0x7fe114152390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe100005210 0x7fe114150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 --2- 172.21.15.33:0/2326048739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe11414ff40 0x7fe11415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 >> 172.21.15.33:0/2326048739 conn(0x7fe114072670 msgr2=0x7fe1140ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 shutdown_connections 2024-05-08T23:32:31.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.685+0000 7fe118bf8700 1 -- 172.21.15.33:0/2326048739 wait complete. 2024-05-08T23:32:31.699 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-05-08T23:32:31.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:31.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.835+0000 7fd7c6a4c700 1 Processor -- start 2024-05-08T23:32:31.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.835+0000 7fd7c6a4c700 1 -- start start 2024-05-08T23:32:31.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7c6a4c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7c006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7c6a4c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c015d870 con 0x7fd7c0154460 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7c6a4c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c0071150 con 0x7fd7c006f810 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7c6a4c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00c7e30 con 0x7fd7c015ad80 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7bffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7c0154460 0x7fd7c015dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48594/0 (socket says 172.21.15.33:48594) 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7bffff700 1 -- 172.21.15.33:0/2894074513 learned_addr learned my addr 172.21.15.33:0/2894074513 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.836+0000 7fd7bf7fe700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7c006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090656304 0 0) 0x7fd7c015d870 con 0x7fd7c0154460 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a80035b0 con 0x7fd7c0154460 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530137243 0 0) 0x7fd7c0071150 con 0x7fd7c006f810 2024-05-08T23:32:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7c015d870 con 0x7fd7c006f810 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192267990 0 0) 0x7fd7a80035b0 con 0x7fd7c0154460 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 msgr2=0x7fd7c006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7c006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 >> v1:172.21.15.42:6789/0 conn(0x7fd7c006f810 legacy=0x7fd7c0164bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c015b220 con 0x7fd7c0154460 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fd7b0001670 con 0x7fd7c0154460 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 >> v1:172.21.15.33:6789/0 conn(0x7fd7c0154460 legacy=0x7fd7c015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8003bc0 0x7fd7a8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7a8004d90 0x7fd7a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00c7e30 con 0x7fd7a8003bc0 2024-05-08T23:32:31.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a800bac0 con 0x7fd7a8004d90 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a800bc30 con 0x7fd7c015ad80 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bffff700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8003bc0 0x7fd7a8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bf7fe700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7c4fe9700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7a8004d90 0x7fd7a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bffff700 1 -- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 msgr2=0x7fd7a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bffff700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7c015ad80 0x7fd7a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bffff700 1 -- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7a8004d90 msgr2=0x7fd7a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.837+0000 7fd7bffff700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7a8004d90 0x7fd7a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7bffff700 1 -- 172.21.15.33:0/2894074513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80035b0 con 0x7fd7a8003bc0 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7c4fe9700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7a8004d90 0x7fd7a8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7bffff700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8003bc0 0x7fd7a8003fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0004ff0 tx=0x7fd7a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad60ae309abc13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0005b60 con 0x7fd7a8003bc0 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0005ce0 con 0x7fd7a8003bc0 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7beffd700 1 -- 172.21.15.33:0/2894074513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0008320 con 0x7fd7a8003bc0 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/2894074513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8003bc0 msgr2=0x7fd7a8003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/2894074513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8003bc0 0x7fd7a8003fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0004ff0 tx=0x7fd7a8005880 comp rx=0 tx=0).stop 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.838+0000 7fd7c4fe9700 1 -- 172.21.15.33:0/2894074513 reap_dead start 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.839+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/2894074513 shutdown_connections 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.839+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/2894074513 >> 172.21.15.33:0/2894074513 conn(0x7fd7c006e990 msgr2=0x7fd7c006ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.839+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/2894074513 shutdown_connections 2024-05-08T23:32:31.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.839+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/2894074513 wait complete. 2024-05-08T23:32:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.840+0000 7fd7c6a4c700 1 Processor -- start 2024-05-08T23:32:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 -- start start 2024-05-08T23:32:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7a8003bc0 0x7fd7c01e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c01e57f0 0x7fd7c00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01e4250 con 0x7fd7c0156630 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01e5530 con 0x7fd7c01e57f0 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.841+0000 7fd7c6a4c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01e3360 con 0x7fd7a8003bc0 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56790/0 (socket says 172.21.15.33:56790) 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 -- 172.21.15.33:0/1159942548 learned_addr learned my addr 172.21.15.33:0/1159942548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7bffff700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7a8003bc0 0x7fd7c01e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7bf7fe700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c01e57f0 0x7fd7c00cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 -- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7a8003bc0 msgr2=0x7fd7c01e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7a8003bc0 0x7fd7c01e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 -- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c01e57f0 msgr2=0x7fd7c00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c01e57f0 0x7fd7c00cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 -- 172.21.15.33:0/1159942548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c01e35d0 con 0x7fd7c0156630 2024-05-08T23:32:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c4fe9700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0157f30 tx=0x7fd7b80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8003c10 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8003d90 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8005310 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c01e38b0 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.842+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01e3e70 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.843+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c00726a0 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.844+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd7b8005470 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.844+0000 7fd7bcff9700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.845+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7b8089340 con 0x7fd7c0156630 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.845+0000 7fd7bffff700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:31.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.845+0000 7fd7bffff700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 0x7fd7ac06d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0004ff0 tx=0x7fd7b000a070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:31.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:31.849+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd7b8054520 con 0x7fd7c0156630 2024-05-08T23:32:32.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.064+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7c0071990 con 0x7fd7ac06ac90 2024-05-08T23:32:32.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.072+0000 7fd7bcff9700 1 -- 172.21.15.33:0/1159942548 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264596 (secure 0 0 0) 0x7fd7b0005e50 con 0x7fd7ac06ac90 2024-05-08T23:32:32.075 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:32.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 msgr2=0x7fd7ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 0x7fd7ac06d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0004ff0 tx=0x7fd7b000a070 comp rx=0 tx=0).stop 2024-05-08T23:32:32.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 msgr2=0x7fd7c0154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0157f30 tx=0x7fd7b80021d0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 shutdown_connections 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd7ac06ac90 0x7fd7ac06d150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7c0156630 0x7fd7c0154ea0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7a8003bc0 0x7fd7c01e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 --2- 172.21.15.33:0/1159942548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7c01e57f0 0x7fd7c00cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 >> 172.21.15.33:0/1159942548 conn(0x7fd7c006e990 msgr2=0x7fd7c015d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 shutdown_connections 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.075+0000 7fd7c6a4c700 1 -- 172.21.15.33:0/1159942548 wait complete. 2024-05-08T23:32:32.078 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:32.089 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-05-08T23:32:30.775602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219048,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003211"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.345654+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:15.345654+0000","last_peered":"2024-05-08T23:32:15.345654+0000","last_clean":"2024-05-08T23:32:15.345654+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:15.345654+0000","last_undegraded":"2024-05-08T23:32:15.345654+0000","last_fullsized":"2024-05-08T23:32:15.345654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.835040+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:13.835040+0000","last_peered":"2024-05-08T23:32:13.835040+0000","last_clean":"2024-05-08T23:32:13.835040+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:13.835040+0000","last_undegraded":"2024-05-08T23:32:13.835040+0000","last_fullsized":"2024-05-08T23:32:13.835040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343181+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:15.343181+0000","last_peered":"2024-05-08T23:32:15.343181+0000","last_clean":"2024-05-08T23:32:15.343181+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:15.343181+0000","last_undegraded":"2024-05-08T23:32:15.343181+0000","last_fullsized":"2024-05-08T23:32:15.343181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343523+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:15.343523+0000","last_peered":"2024-05-08T23:32:15.343523+0000","last_clean":"2024-05-08T23:32:15.343523+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:15.343523+0000","last_undegraded":"2024-05-08T23:32:15.343523+0000","last_fullsized":"2024-05-08T23:32:15.343523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.834996+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:13.834996+0000","last_peered":"2024-05-08T23:32:13.834996+0000","last_clean":"2024-05-08T23:32:13.834996+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:13.834996+0000","last_undegraded":"2024-05-08T23:32:13.834996+0000","last_fullsized":"2024-05-08T23:32:13.834996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839936+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:13.839936+0000","last_peered":"2024-05-08T23:32:13.839936+0000","last_clean":"2024-05-08T23:32:13.839936+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:13.839936+0000","last_undegraded":"2024-05-08T23:32:13.839936+0000","last_fullsized":"2024-05-08T23:32:13.839936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342601+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:15.342601+0000","last_peered":"2024-05-08T23:32:15.342601+0000","last_clean":"2024-05-08T23:32:15.342601+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:15.342601+0000","last_undegraded":"2024-05-08T23:32:15.342601+0000","last_fullsized":"2024-05-08T23:32:15.342601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.344741+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:15.344741+0000","last_peered":"2024-05-08T23:32:15.344741+0000","last_clean":"2024-05-08T23:32:15.344741+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:15.344741+0000","last_undegraded":"2024-05-08T23:32:15.344741+0000","last_fullsized":"2024-05-08T23:32:15.344741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342373+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:15.342373+0000","last_peered":"2024-05-08T23:32:15.342373+0000","last_clean":"2024-05-08T23:32:15.342373+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:15.342373+0000","last_undegraded":"2024-05-08T23:32:15.342373+0000","last_fullsized":"2024-05-08T23:32:15.342373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.350409+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:15.350409+0000","last_peered":"2024-05-08T23:32:15.350409+0000","last_clean":"2024-05-08T23:32:15.350409+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:15.350409+0000","last_undegraded":"2024-05-08T23:32:15.350409+0000","last_fullsized":"2024-05-08T23:32:15.350409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.841119+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:13.841119+0000","last_peered":"2024-05-08T23:32:13.841119+0000","last_clean":"2024-05-08T23:32:13.841119+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:13.841119+0000","last_undegraded":"2024-05-08T23:32:13.841119+0000","last_fullsized":"2024-05-08T23:32:13.841119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839387+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:13.839387+0000","last_peered":"2024-05-08T23:32:13.839387+0000","last_clean":"2024-05-08T23:32:13.839387+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:13.839387+0000","last_undegraded":"2024-05-08T23:32:13.839387+0000","last_fullsized":"2024-05-08T23:32:13.839387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342603+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:15.342603+0000","last_peered":"2024-05-08T23:32:15.342603+0000","last_clean":"2024-05-08T23:32:15.342603+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:15.342603+0000","last_undegraded":"2024-05-08T23:32:15.342603+0000","last_fullsized":"2024-05-08T23:32:15.342603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342739+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:15.342739+0000","last_peered":"2024-05-08T23:32:15.342739+0000","last_clean":"2024-05-08T23:32:15.342739+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:15.342739+0000","last_undegraded":"2024-05-08T23:32:15.342739+0000","last_fullsized":"2024-05-08T23:32:15.342739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839350+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:13.839350+0000","last_peered":"2024-05-08T23:32:13.839350+0000","last_clean":"2024-05-08T23:32:13.839350+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:13.839350+0000","last_undegraded":"2024-05-08T23:32:13.839350+0000","last_fullsized":"2024-05-08T23:32:13.839350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343951+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:15.343951+0000","last_peered":"2024-05-08T23:32:15.343951+0000","last_clean":"2024-05-08T23:32:15.343951+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:15.343951+0000","last_undegraded":"2024-05-08T23:32:15.343951+0000","last_fullsized":"2024-05-08T23:32:15.343951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:32.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:32:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.268+0000 7fdefa9bb700 1 Processor -- start 2024-05-08T23:32:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.268+0000 7fdefa9bb700 1 -- start start 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdefa9bb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdef40b1ad0 0x7fdef40b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdefa9bb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef40711c0 con 0x7fdef40c9220 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdefa9bb700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef4071710 con 0x7fdef415b360 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdefa9bb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef406b0a0 con 0x7fdef40b1ad0 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdef8f58700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdef40b1ad0 0x7fdef40b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdef37fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdef40c9220 0x7fdef40729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48600/0 (socket says 172.21.15.33:48600) 2024-05-08T23:32:32.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.269+0000 7fdef37fe700 1 -- 172.21.15.33:0/4253313158 learned_addr learned my addr 172.21.15.33:0/4253313158 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef8f58700 1 -- 172.21.15.33:0/4253313158 >> v1:172.21.15.42:6789/0 conn(0x7fdef415b360 legacy=0x7fdef406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef8f58700 1 -- 172.21.15.33:0/4253313158 >> v1:172.21.15.33:6789/0 conn(0x7fdef40c9220 legacy=0x7fdef40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef2ffd700 1 -- 172.21.15.33:0/4253313158 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299028472 0 0) 0x7fdef40711c0 con 0x7fdef40c9220 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef8f58700 1 -- 172.21.15.33:0/4253313158 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4071430 con 0x7fdef40b1ad0 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef8f58700 1 --2- 172.21.15.33:0/4253313158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40b1ad0 0x7fdef40b1f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d90 tx=0x7fdeec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0db065dd3d042a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef2ffd700 1 -- 172.21.15.33:0/4253313158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00e070 con 0x7fdef40b1ad0 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef2ffd700 1 -- 172.21.15.33:0/4253313158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00bbd0 con 0x7fdef40b1ad0 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.270+0000 7fdef2ffd700 1 -- 172.21.15.33:0/4253313158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec012430 con 0x7fdef40b1ad0 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 -- 172.21.15.33:0/4253313158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40b1ad0 msgr2=0x7fdef40b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/4253313158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40b1ad0 0x7fdef40b1f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d90 tx=0x7fdeec00b330 comp rx=0 tx=0).stop 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 -- 172.21.15.33:0/4253313158 shutdown_connections 2024-05-08T23:32:32.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/4253313158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40b1ad0 0x7fdef40b1f00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 -- 172.21.15.33:0/4253313158 >> 172.21.15.33:0/4253313158 conn(0x7fdef406d2b0 msgr2=0x7fdef406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 -- 172.21.15.33:0/4253313158 shutdown_connections 2024-05-08T23:32:32.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.271+0000 7fdefa9bb700 1 -- 172.21.15.33:0/4253313158 wait complete. 2024-05-08T23:32:32.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.272+0000 7fdefa9bb700 1 Processor -- start 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.272+0000 7fdefa9bb700 1 -- start start 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdef40b1ad0 0x7fdef40c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdef415b360 0x7fdef40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40c7040 con 0x7fdef40c9220 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40aeac0 con 0x7fdef415b360 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdefa9bb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40c6d60 con 0x7fdef40b1ad0 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56804/0 (socket says 172.21.15.33:56804) 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef37fe700 1 -- 172.21.15.33:0/219304665 learned_addr learned my addr 172.21.15.33:0/219304665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:32.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef8f58700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdef415b360 0x7fdef40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef3fff700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdef40b1ad0 0x7fdef40c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef37fe700 1 -- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdef40b1ad0 msgr2=0x7fdef40c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.273+0000 7fdef37fe700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdef40b1ad0 0x7fdef40c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef37fe700 1 -- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdef415b360 msgr2=0x7fdef40af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef37fe700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdef415b360 0x7fdef40af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef37fe700 1 -- 172.21.15.33:0/219304665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec009a70 con 0x7fdef40c9220 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef37fe700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c520 tx=0x7fdee800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004c80 con 0x7fdef40c9220 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8004e00 con 0x7fdef40c9220 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8007310 con 0x7fdef40c9220 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef40c4c80 con 0x7fdef40c9220 2024-05-08T23:32:32.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.274+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef40c5240 con 0x7fdef40c9220 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.275+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef40aec00 con 0x7fdef40c9220 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.276+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdee8007470 con 0x7fdef40c9220 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.276+0000 7fdef0ff9700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 0x7fdee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.276+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee8088fe0 con 0x7fdef40c9220 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.276+0000 7fdef3fff700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 0x7fdee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.277+0000 7fdef3fff700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 0x7fdee006d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdee4000f60 tx=0x7fdee4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.281+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdef40aec00 con 0x7fdef40c9220 2024-05-08T23:32:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.494+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdef406c970 con 0x7fdee006ac90 2024-05-08T23:32:32.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.502+0000 7fdef0ff9700 1 -- 172.21.15.33:0/219304665 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264596 (secure 0 0 0) 0x7fdee4001370 con 0x7fdee006ac90 2024-05-08T23:32:32.505 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 msgr2=0x7fdee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 0x7fdee006d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdee4000f60 tx=0x7fdee4009040 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 msgr2=0x7fdef40aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c520 tx=0x7fdee800c8e0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 shutdown_connections 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee006ac90 0x7fdee006d150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdef415b360 0x7fdef40af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdef40b1ad0 0x7fdef40c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 --2- 172.21.15.33:0/219304665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef40c9220 0x7fdef40aeee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 >> 172.21.15.33:0/219304665 conn(0x7fdef406d2b0 msgr2=0x7fdef406d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 shutdown_connections 2024-05-08T23:32:32.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.506+0000 7fdefa9bb700 1 -- 172.21.15.33:0/219304665 wait complete. 2024-05-08T23:32:32.509 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:32:32.520 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-05-08T23:32:30.775602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219048,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003211"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680708+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:32:11.680708+0000","last_peered":"2024-05-08T23:32:11.680708+0000","last_clean":"2024-05-08T23:32:11.680708+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:32:11.680708+0000","last_undegraded":"2024-05-08T23:32:11.680708+0000","last_fullsized":"2024-05-08T23:32:11.680708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341177+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:32:11.341177+0000","last_peered":"2024-05-08T23:32:11.341177+0000","last_clean":"2024-05-08T23:32:11.341177+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:32:11.341177+0000","last_undegraded":"2024-05-08T23:32:11.341177+0000","last_fullsized":"2024-05-08T23:32:11.341177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:32:11.679895+0000","last_peered":"2024-05-08T23:32:11.679895+0000","last_clean":"2024-05-08T23:32:11.679895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:32:11.679895+0000","last_undegraded":"2024-05-08T23:32:11.679895+0000","last_fullsized":"2024-05-08T23:32:11.679895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342072+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:32:11.342072+0000","last_peered":"2024-05-08T23:32:11.342072+0000","last_clean":"2024-05-08T23:32:11.342072+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:32:11.342072+0000","last_undegraded":"2024-05-08T23:32:11.342072+0000","last_fullsized":"2024-05-08T23:32:11.342072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348821+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.348499+0000","last_became_peered":"2024-05-08T23:32:10.348499+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:27.513214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341805+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:32:11.341805+0000","last_peered":"2024-05-08T23:32:11.341805+0000","last_clean":"2024-05-08T23:32:11.341805+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:32:11.341805+0000","last_undegraded":"2024-05-08T23:32:11.341805+0000","last_fullsized":"2024-05-08T23:32:11.341805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679766+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:32:11.679766+0000","last_peered":"2024-05-08T23:32:11.679766+0000","last_clean":"2024-05-08T23:32:11.679766+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:32:11.679766+0000","last_undegraded":"2024-05-08T23:32:11.679766+0000","last_fullsized":"2024-05-08T23:32:11.679766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680117+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:32:11.680117+0000","last_peered":"2024-05-08T23:32:11.680117+0000","last_clean":"2024-05-08T23:32:11.680117+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:32:11.680117+0000","last_undegraded":"2024-05-08T23:32:11.680117+0000","last_fullsized":"2024-05-08T23:32:11.680117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680643+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:32:11.680643+0000","last_peered":"2024-05-08T23:32:11.680643+0000","last_clean":"2024-05-08T23:32:11.680643+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:32:11.680643+0000","last_undegraded":"2024-05-08T23:32:11.680643+0000","last_fullsized":"2024-05-08T23:32:11.680643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680764+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:32:11.680764+0000","last_peered":"2024-05-08T23:32:11.680764+0000","last_clean":"2024-05-08T23:32:11.680764+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:32:11.680764+0000","last_undegraded":"2024-05-08T23:32:11.680764+0000","last_fullsized":"2024-05-08T23:32:11.680764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680621+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:32:11.680621+0000","last_peered":"2024-05-08T23:32:11.680621+0000","last_clean":"2024-05-08T23:32:11.680621+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:32:11.680621+0000","last_undegraded":"2024-05-08T23:32:11.680621+0000","last_fullsized":"2024-05-08T23:32:11.680621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553322+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:32:11.553322+0000","last_peered":"2024-05-08T23:32:11.553322+0000","last_clean":"2024-05-08T23:32:11.553322+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:32:11.553322+0000","last_undegraded":"2024-05-08T23:32:11.553322+0000","last_fullsized":"2024-05-08T23:32:11.553322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680266+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:32:11.680266+0000","last_peered":"2024-05-08T23:32:11.680266+0000","last_clean":"2024-05-08T23:32:11.680266+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:32:11.680266+0000","last_undegraded":"2024-05-08T23:32:11.680266+0000","last_fullsized":"2024-05-08T23:32:11.680266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679638+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:32:11.679638+0000","last_peered":"2024-05-08T23:32:11.679638+0000","last_clean":"2024-05-08T23:32:11.679638+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:32:11.679638+0000","last_undegraded":"2024-05-08T23:32:11.679638+0000","last_fullsized":"2024-05-08T23:32:11.679638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340889+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:32:11.340889+0000","last_peered":"2024-05-08T23:32:11.340889+0000","last_clean":"2024-05-08T23:32:11.340889+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:32:11.340889+0000","last_undegraded":"2024-05-08T23:32:11.340889+0000","last_fullsized":"2024-05-08T23:32:11.340889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341687+0000","last_change":"2024-05-08T23:32:10.348068+0000","last_active":"2024-05-08T23:32:11.341687+0000","last_peered":"2024-05-08T23:32:11.341687+0000","last_clean":"2024-05-08T23:32:11.341687+0000","last_became_active":"2024-05-08T23:32:10.347821+0000","last_became_peered":"2024-05-08T23:32:10.347821+0000","last_unstale":"2024-05-08T23:32:11.341687+0000","last_undegraded":"2024-05-08T23:32:11.341687+0000","last_fullsized":"2024-05-08T23:32:11.341687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:46.966206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342296+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:32:11.342296+0000","last_peered":"2024-05-08T23:32:11.342296+0000","last_clean":"2024-05-08T23:32:11.342296+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:32:11.342296+0000","last_undegraded":"2024-05-08T23:32:11.342296+0000","last_fullsized":"2024-05-08T23:32:11.342296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679362+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:32:11.679362+0000","last_peered":"2024-05-08T23:32:11.679362+0000","last_clean":"2024-05-08T23:32:11.679362+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:32:11.679362+0000","last_undegraded":"2024-05-08T23:32:11.679362+0000","last_fullsized":"2024-05-08T23:32:11.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680343+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:32:11.680343+0000","last_peered":"2024-05-08T23:32:11.680343+0000","last_clean":"2024-05-08T23:32:11.680343+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:32:11.680343+0000","last_undegraded":"2024-05-08T23:32:11.680343+0000","last_fullsized":"2024-05-08T23:32:11.680343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.345654+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:32:15.345654+0000","last_peered":"2024-05-08T23:32:15.345654+0000","last_clean":"2024-05-08T23:32:15.345654+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:32:15.345654+0000","last_undegraded":"2024-05-08T23:32:15.345654+0000","last_fullsized":"2024-05-08T23:32:15.345654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680645+0000","last_change":"2024-05-08T23:32:10.352549+0000","last_active":"2024-05-08T23:32:11.680645+0000","last_peered":"2024-05-08T23:32:11.680645+0000","last_clean":"2024-05-08T23:32:11.680645+0000","last_became_active":"2024-05-08T23:32:10.352277+0000","last_became_peered":"2024-05-08T23:32:10.352277+0000","last_unstale":"2024-05-08T23:32:11.680645+0000","last_undegraded":"2024-05-08T23:32:11.680645+0000","last_fullsized":"2024-05-08T23:32:11.680645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:07.164668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.835040+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:32:13.835040+0000","last_peered":"2024-05-08T23:32:13.835040+0000","last_clean":"2024-05-08T23:32:13.835040+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:32:13.835040+0000","last_undegraded":"2024-05-08T23:32:13.835040+0000","last_fullsized":"2024-05-08T23:32:13.835040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.353416+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.353104+0000","last_became_peered":"2024-05-08T23:32:10.353104+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:35.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343181+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:32:15.343181+0000","last_peered":"2024-05-08T23:32:15.343181+0000","last_clean":"2024-05-08T23:32:15.343181+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:32:15.343181+0000","last_undegraded":"2024-05-08T23:32:15.343181+0000","last_fullsized":"2024-05-08T23:32:15.343181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342253+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:32:11.342253+0000","last_peered":"2024-05-08T23:32:11.342253+0000","last_clean":"2024-05-08T23:32:11.342253+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:32:11.342253+0000","last_undegraded":"2024-05-08T23:32:11.342253+0000","last_fullsized":"2024-05-08T23:32:11.342253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343523+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:32:15.343523+0000","last_peered":"2024-05-08T23:32:15.343523+0000","last_clean":"2024-05-08T23:32:15.343523+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:32:15.343523+0000","last_undegraded":"2024-05-08T23:32:15.343523+0000","last_fullsized":"2024-05-08T23:32:15.343523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342291+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:32:11.342291+0000","last_peered":"2024-05-08T23:32:11.342291+0000","last_clean":"2024-05-08T23:32:11.342291+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:32:11.342291+0000","last_undegraded":"2024-05-08T23:32:11.342291+0000","last_fullsized":"2024-05-08T23:32:11.342291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.834996+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:32:13.834996+0000","last_peered":"2024-05-08T23:32:13.834996+0000","last_clean":"2024-05-08T23:32:13.834996+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:32:13.834996+0000","last_undegraded":"2024-05-08T23:32:13.834996+0000","last_fullsized":"2024-05-08T23:32:13.834996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.552972+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:32:11.552972+0000","last_peered":"2024-05-08T23:32:11.552972+0000","last_clean":"2024-05-08T23:32:11.552972+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:32:11.552972+0000","last_undegraded":"2024-05-08T23:32:11.552972+0000","last_fullsized":"2024-05-08T23:32:11.552972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839936+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:32:13.839936+0000","last_peered":"2024-05-08T23:32:13.839936+0000","last_clean":"2024-05-08T23:32:13.839936+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:32:13.839936+0000","last_undegraded":"2024-05-08T23:32:13.839936+0000","last_fullsized":"2024-05-08T23:32:13.839936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680154+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:32:11.680154+0000","last_peered":"2024-05-08T23:32:11.680154+0000","last_clean":"2024-05-08T23:32:11.680154+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:32:11.680154+0000","last_undegraded":"2024-05-08T23:32:11.680154+0000","last_fullsized":"2024-05-08T23:32:11.680154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681255+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:32:11.681255+0000","last_peered":"2024-05-08T23:32:11.681255+0000","last_clean":"2024-05-08T23:32:11.681255+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:32:11.681255+0000","last_undegraded":"2024-05-08T23:32:11.681255+0000","last_fullsized":"2024-05-08T23:32:11.681255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680226+0000","last_change":"2024-05-08T23:32:10.351977+0000","last_active":"2024-05-08T23:32:11.680226+0000","last_peered":"2024-05-08T23:32:11.680226+0000","last_clean":"2024-05-08T23:32:11.680226+0000","last_became_active":"2024-05-08T23:32:10.351656+0000","last_became_peered":"2024-05-08T23:32:10.351656+0000","last_unstale":"2024-05-08T23:32:11.680226+0000","last_undegraded":"2024-05-08T23:32:11.680226+0000","last_fullsized":"2024-05-08T23:32:11.680226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:10.060751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341897+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:32:11.341897+0000","last_peered":"2024-05-08T23:32:11.341897+0000","last_clean":"2024-05-08T23:32:11.341897+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:32:11.341897+0000","last_undegraded":"2024-05-08T23:32:11.341897+0000","last_fullsized":"2024-05-08T23:32:11.341897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341011+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:32:11.341011+0000","last_peered":"2024-05-08T23:32:11.341011+0000","last_clean":"2024-05-08T23:32:11.341011+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:32:11.341011+0000","last_undegraded":"2024-05-08T23:32:11.341011+0000","last_fullsized":"2024-05-08T23:32:11.341011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680169+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:32:11.680169+0000","last_peered":"2024-05-08T23:32:11.680169+0000","last_clean":"2024-05-08T23:32:11.680169+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:32:11.680169+0000","last_undegraded":"2024-05-08T23:32:11.680169+0000","last_fullsized":"2024-05-08T23:32:11.680169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342186+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:32:11.342186+0000","last_peered":"2024-05-08T23:32:11.342186+0000","last_clean":"2024-05-08T23:32:11.342186+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:32:11.342186+0000","last_undegraded":"2024-05-08T23:32:11.342186+0000","last_fullsized":"2024-05-08T23:32:11.342186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342020+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:32:11.342020+0000","last_peered":"2024-05-08T23:32:11.342020+0000","last_clean":"2024-05-08T23:32:11.342020+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:32:11.342020+0000","last_undegraded":"2024-05-08T23:32:11.342020+0000","last_fullsized":"2024-05-08T23:32:11.342020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.353094+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.352809+0000","last_became_peered":"2024-05-08T23:32:10.352809+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:47.292116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342601+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:32:15.342601+0000","last_peered":"2024-05-08T23:32:15.342601+0000","last_clean":"2024-05-08T23:32:15.342601+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:32:15.342601+0000","last_undegraded":"2024-05-08T23:32:15.342601+0000","last_fullsized":"2024-05-08T23:32:15.342601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553261+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:32:11.553261+0000","last_peered":"2024-05-08T23:32:11.553261+0000","last_clean":"2024-05-08T23:32:11.553261+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:32:11.553261+0000","last_undegraded":"2024-05-08T23:32:11.553261+0000","last_fullsized":"2024-05-08T23:32:11.553261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.344741+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:32:15.344741+0000","last_peered":"2024-05-08T23:32:15.344741+0000","last_clean":"2024-05-08T23:32:15.344741+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:32:15.344741+0000","last_undegraded":"2024-05-08T23:32:15.344741+0000","last_fullsized":"2024-05-08T23:32:15.344741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680459+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:32:11.680459+0000","last_peered":"2024-05-08T23:32:11.680459+0000","last_clean":"2024-05-08T23:32:11.680459+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:32:11.680459+0000","last_undegraded":"2024-05-08T23:32:11.680459+0000","last_fullsized":"2024-05-08T23:32:11.680459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342373+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:32:15.342373+0000","last_peered":"2024-05-08T23:32:15.342373+0000","last_clean":"2024-05-08T23:32:15.342373+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:32:15.342373+0000","last_undegraded":"2024-05-08T23:32:15.342373+0000","last_fullsized":"2024-05-08T23:32:15.342373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679602+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:32:11.679602+0000","last_peered":"2024-05-08T23:32:11.679602+0000","last_clean":"2024-05-08T23:32:11.679602+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:32:11.679602+0000","last_undegraded":"2024-05-08T23:32:11.679602+0000","last_fullsized":"2024-05-08T23:32:11.679602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342202+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:32:11.342202+0000","last_peered":"2024-05-08T23:32:11.342202+0000","last_clean":"2024-05-08T23:32:11.342202+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:32:11.342202+0000","last_undegraded":"2024-05-08T23:32:11.342202+0000","last_fullsized":"2024-05-08T23:32:11.342202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680202+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:32:11.680202+0000","last_peered":"2024-05-08T23:32:11.680202+0000","last_clean":"2024-05-08T23:32:11.680202+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:32:11.680202+0000","last_undegraded":"2024-05-08T23:32:11.680202+0000","last_fullsized":"2024-05-08T23:32:11.680202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680013+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:32:11.680013+0000","last_peered":"2024-05-08T23:32:11.680013+0000","last_clean":"2024-05-08T23:32:11.680013+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:32:11.680013+0000","last_undegraded":"2024-05-08T23:32:11.680013+0000","last_fullsized":"2024-05-08T23:32:11.680013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679778+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:32:11.679778+0000","last_peered":"2024-05-08T23:32:11.679778+0000","last_clean":"2024-05-08T23:32:11.679778+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:32:11.679778+0000","last_undegraded":"2024-05-08T23:32:11.679778+0000","last_fullsized":"2024-05-08T23:32:11.679778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.350409+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:32:15.350409+0000","last_peered":"2024-05-08T23:32:15.350409+0000","last_clean":"2024-05-08T23:32:15.350409+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:32:15.350409+0000","last_undegraded":"2024-05-08T23:32:15.350409+0000","last_fullsized":"2024-05-08T23:32:15.350409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679898+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:32:11.679898+0000","last_peered":"2024-05-08T23:32:11.679898+0000","last_clean":"2024-05-08T23:32:11.679898+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:32:11.679898+0000","last_undegraded":"2024-05-08T23:32:11.679898+0000","last_fullsized":"2024-05-08T23:32:11.679898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.841119+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:32:13.841119+0000","last_peered":"2024-05-08T23:32:13.841119+0000","last_clean":"2024-05-08T23:32:13.841119+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:32:13.841119+0000","last_undegraded":"2024-05-08T23:32:13.841119+0000","last_fullsized":"2024-05-08T23:32:13.841119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680561+0000","last_change":"2024-05-08T23:32:10.354316+0000","last_active":"2024-05-08T23:32:11.680561+0000","last_peered":"2024-05-08T23:32:11.680561+0000","last_clean":"2024-05-08T23:32:11.680561+0000","last_became_active":"2024-05-08T23:32:10.353824+0000","last_became_peered":"2024-05-08T23:32:10.353824+0000","last_unstale":"2024-05-08T23:32:11.680561+0000","last_undegraded":"2024-05-08T23:32:11.680561+0000","last_fullsized":"2024-05-08T23:32:11.680561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:23.366718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340987+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:32:11.340987+0000","last_peered":"2024-05-08T23:32:11.340987+0000","last_clean":"2024-05-08T23:32:11.340987+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:32:11.340987+0000","last_undegraded":"2024-05-08T23:32:11.340987+0000","last_fullsized":"2024-05-08T23:32:11.340987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342379+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:32:11.342379+0000","last_peered":"2024-05-08T23:32:11.342379+0000","last_clean":"2024-05-08T23:32:11.342379+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:32:11.342379+0000","last_undegraded":"2024-05-08T23:32:11.342379+0000","last_fullsized":"2024-05-08T23:32:11.342379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342177+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:32:11.342177+0000","last_peered":"2024-05-08T23:32:11.342177+0000","last_clean":"2024-05-08T23:32:11.342177+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:32:11.342177+0000","last_undegraded":"2024-05-08T23:32:11.342177+0000","last_fullsized":"2024-05-08T23:32:11.342177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553205+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:32:11.553205+0000","last_peered":"2024-05-08T23:32:11.553205+0000","last_clean":"2024-05-08T23:32:11.553205+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:32:11.553205+0000","last_undegraded":"2024-05-08T23:32:11.553205+0000","last_fullsized":"2024-05-08T23:32:11.553205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340915+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:32:11.340915+0000","last_peered":"2024-05-08T23:32:11.340915+0000","last_clean":"2024-05-08T23:32:11.340915+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:32:11.340915+0000","last_undegraded":"2024-05-08T23:32:11.340915+0000","last_fullsized":"2024-05-08T23:32:11.340915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679274+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:32:11.679274+0000","last_peered":"2024-05-08T23:32:11.679274+0000","last_clean":"2024-05-08T23:32:11.679274+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:32:11.679274+0000","last_undegraded":"2024-05-08T23:32:11.679274+0000","last_fullsized":"2024-05-08T23:32:11.679274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680381+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:32:11.680381+0000","last_peered":"2024-05-08T23:32:11.680381+0000","last_clean":"2024-05-08T23:32:11.680381+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:32:11.680381+0000","last_undegraded":"2024-05-08T23:32:11.680381+0000","last_fullsized":"2024-05-08T23:32:11.680381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341834+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:32:11.341834+0000","last_peered":"2024-05-08T23:32:11.341834+0000","last_clean":"2024-05-08T23:32:11.341834+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:32:11.341834+0000","last_undegraded":"2024-05-08T23:32:11.341834+0000","last_fullsized":"2024-05-08T23:32:11.341834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-08T23:32:10.338959+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:32:10.338959+0000","last_peered":"2024-05-08T23:32:10.338959+0000","last_clean":"2024-05-08T23:32:10.338959+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:32:10.338959+0000","last_undegraded":"2024-05-08T23:32:10.338959+0000","last_fullsized":"2024-05-08T23:32:10.338959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679498+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:32:11.679498+0000","last_peered":"2024-05-08T23:32:11.679498+0000","last_clean":"2024-05-08T23:32:11.679498+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:32:11.679498+0000","last_undegraded":"2024-05-08T23:32:11.679498+0000","last_fullsized":"2024-05-08T23:32:11.679498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679830+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:32:11.679830+0000","last_peered":"2024-05-08T23:32:11.679830+0000","last_clean":"2024-05-08T23:32:11.679830+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:32:11.679830+0000","last_undegraded":"2024-05-08T23:32:11.679830+0000","last_fullsized":"2024-05-08T23:32:11.679830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680568+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:32:11.680568+0000","last_peered":"2024-05-08T23:32:11.680568+0000","last_clean":"2024-05-08T23:32:11.680568+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:32:11.680568+0000","last_undegraded":"2024-05-08T23:32:11.680568+0000","last_fullsized":"2024-05-08T23:32:11.680568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341266+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:32:11.341266+0000","last_peered":"2024-05-08T23:32:11.341266+0000","last_clean":"2024-05-08T23:32:11.341266+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:32:11.341266+0000","last_undegraded":"2024-05-08T23:32:11.341266+0000","last_fullsized":"2024-05-08T23:32:11.341266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839387+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:32:13.839387+0000","last_peered":"2024-05-08T23:32:13.839387+0000","last_clean":"2024-05-08T23:32:13.839387+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:32:13.839387+0000","last_undegraded":"2024-05-08T23:32:13.839387+0000","last_fullsized":"2024-05-08T23:32:13.839387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680652+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:32:11.680652+0000","last_peered":"2024-05-08T23:32:11.680652+0000","last_clean":"2024-05-08T23:32:11.680652+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:32:11.680652+0000","last_undegraded":"2024-05-08T23:32:11.680652+0000","last_fullsized":"2024-05-08T23:32:11.680652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342603+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:32:15.342603+0000","last_peered":"2024-05-08T23:32:15.342603+0000","last_clean":"2024-05-08T23:32:15.342603+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:32:15.342603+0000","last_undegraded":"2024-05-08T23:32:15.342603+0000","last_fullsized":"2024-05-08T23:32:15.342603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679814+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:32:11.679814+0000","last_peered":"2024-05-08T23:32:11.679814+0000","last_clean":"2024-05-08T23:32:11.679814+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:32:11.679814+0000","last_undegraded":"2024-05-08T23:32:11.679814+0000","last_fullsized":"2024-05-08T23:32:11.679814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679713+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:32:11.679713+0000","last_peered":"2024-05-08T23:32:11.679713+0000","last_clean":"2024-05-08T23:32:11.679713+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:32:11.679713+0000","last_undegraded":"2024-05-08T23:32:11.679713+0000","last_fullsized":"2024-05-08T23:32:11.679713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.553085+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:32:11.553085+0000","last_peered":"2024-05-08T23:32:11.553085+0000","last_clean":"2024-05-08T23:32:11.553085+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:32:11.553085+0000","last_undegraded":"2024-05-08T23:32:11.553085+0000","last_fullsized":"2024-05-08T23:32:11.553085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.340883+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:32:11.340883+0000","last_peered":"2024-05-08T23:32:11.340883+0000","last_clean":"2024-05-08T23:32:11.340883+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:32:11.340883+0000","last_undegraded":"2024-05-08T23:32:11.340883+0000","last_fullsized":"2024-05-08T23:32:11.340883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680203+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:32:11.680203+0000","last_peered":"2024-05-08T23:32:11.680203+0000","last_clean":"2024-05-08T23:32:11.680203+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:32:11.680203+0000","last_undegraded":"2024-05-08T23:32:11.680203+0000","last_fullsized":"2024-05-08T23:32:11.680203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341033+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:32:11.341033+0000","last_peered":"2024-05-08T23:32:11.341033+0000","last_clean":"2024-05-08T23:32:11.341033+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:32:11.341033+0000","last_undegraded":"2024-05-08T23:32:11.341033+0000","last_fullsized":"2024-05-08T23:32:11.341033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680630+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:32:11.680630+0000","last_peered":"2024-05-08T23:32:11.680630+0000","last_clean":"2024-05-08T23:32:11.680630+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:32:11.680630+0000","last_undegraded":"2024-05-08T23:32:11.680630+0000","last_fullsized":"2024-05-08T23:32:11.680630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.342739+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:32:15.342739+0000","last_peered":"2024-05-08T23:32:15.342739+0000","last_clean":"2024-05-08T23:32:15.342739+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:32:15.342739+0000","last_undegraded":"2024-05-08T23:32:15.342739+0000","last_fullsized":"2024-05-08T23:32:15.342739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679931+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:32:11.679931+0000","last_peered":"2024-05-08T23:32:11.679931+0000","last_clean":"2024-05-08T23:32:11.679931+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:32:11.679931+0000","last_undegraded":"2024-05-08T23:32:11.679931+0000","last_fullsized":"2024-05-08T23:32:11.679931+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679376+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:32:11.679376+0000","last_peered":"2024-05-08T23:32:11.679376+0000","last_clean":"2024-05-08T23:32:11.679376+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:32:11.679376+0000","last_undegraded":"2024-05-08T23:32:11.679376+0000","last_fullsized":"2024-05-08T23:32:11.679376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680497+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:32:11.680497+0000","last_peered":"2024-05-08T23:32:11.680497+0000","last_clean":"2024-05-08T23:32:11.680497+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:32:11.680497+0000","last_undegraded":"2024-05-08T23:32:11.680497+0000","last_fullsized":"2024-05-08T23:32:11.680497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:13.839350+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:32:13.839350+0000","last_peered":"2024-05-08T23:32:13.839350+0000","last_clean":"2024-05-08T23:32:13.839350+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:32:13.839350+0000","last_undegraded":"2024-05-08T23:32:13.839350+0000","last_fullsized":"2024-05-08T23:32:13.839350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342319+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:32:11.342319+0000","last_peered":"2024-05-08T23:32:11.342319+0000","last_clean":"2024-05-08T23:32:11.342319+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:32:11.342319+0000","last_undegraded":"2024-05-08T23:32:11.342319+0000","last_fullsized":"2024-05-08T23:32:11.342319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:15.343951+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:32:15.343951+0000","last_peered":"2024-05-08T23:32:15.343951+0000","last_clean":"2024-05-08T23:32:15.343951+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:32:15.343951+0000","last_undegraded":"2024-05-08T23:32:15.343951+0000","last_fullsized":"2024-05-08T23:32:15.343951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680773+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:32:11.680773+0000","last_peered":"2024-05-08T23:32:11.680773+0000","last_clean":"2024-05-08T23:32:11.680773+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:32:11.680773+0000","last_undegraded":"2024-05-08T23:32:11.680773+0000","last_fullsized":"2024-05-08T23:32:11.680773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680070+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:32:11.680070+0000","last_peered":"2024-05-08T23:32:11.680070+0000","last_clean":"2024-05-08T23:32:11.680070+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:32:11.680070+0000","last_undegraded":"2024-05-08T23:32:11.680070+0000","last_fullsized":"2024-05-08T23:32:11.680070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679338+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:32:11.679338+0000","last_peered":"2024-05-08T23:32:11.679338+0000","last_clean":"2024-05-08T23:32:11.679338+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:32:11.679338+0000","last_undegraded":"2024-05-08T23:32:11.679338+0000","last_fullsized":"2024-05-08T23:32:11.679338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342043+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:32:11.342043+0000","last_peered":"2024-05-08T23:32:11.342043+0000","last_clean":"2024-05-08T23:32:11.342043+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:32:11.342043+0000","last_undegraded":"2024-05-08T23:32:11.342043+0000","last_fullsized":"2024-05-08T23:32:11.342043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680041+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:32:11.680041+0000","last_peered":"2024-05-08T23:32:11.680041+0000","last_clean":"2024-05-08T23:32:11.680041+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:32:11.680041+0000","last_undegraded":"2024-05-08T23:32:11.680041+0000","last_fullsized":"2024-05-08T23:32:11.680041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680389+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:32:11.680389+0000","last_peered":"2024-05-08T23:32:11.680389+0000","last_clean":"2024-05-08T23:32:11.680389+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:32:11.680389+0000","last_undegraded":"2024-05-08T23:32:11.680389+0000","last_fullsized":"2024-05-08T23:32:11.680389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341095+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:32:11.341095+0000","last_peered":"2024-05-08T23:32:11.341095+0000","last_clean":"2024-05-08T23:32:11.341095+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:32:11.341095+0000","last_undegraded":"2024-05-08T23:32:11.341095+0000","last_fullsized":"2024-05-08T23:32:11.341095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680712+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:32:11.680712+0000","last_peered":"2024-05-08T23:32:11.680712+0000","last_clean":"2024-05-08T23:32:11.680712+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:32:11.680712+0000","last_undegraded":"2024-05-08T23:32:11.680712+0000","last_fullsized":"2024-05-08T23:32:11.680712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680528+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:32:11.680528+0000","last_peered":"2024-05-08T23:32:11.680528+0000","last_clean":"2024-05-08T23:32:11.680528+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:32:11.680528+0000","last_undegraded":"2024-05-08T23:32:11.680528+0000","last_fullsized":"2024-05-08T23:32:11.680528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.342085+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:32:11.342085+0000","last_peered":"2024-05-08T23:32:11.342085+0000","last_clean":"2024-05-08T23:32:11.342085+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:32:11.342085+0000","last_undegraded":"2024-05-08T23:32:11.342085+0000","last_fullsized":"2024-05-08T23:32:11.342085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.681206+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:32:11.681206+0000","last_peered":"2024-05-08T23:32:11.681206+0000","last_clean":"2024-05-08T23:32:11.681206+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:32:11.681206+0000","last_undegraded":"2024-05-08T23:32:11.681206+0000","last_fullsized":"2024-05-08T23:32:11.681206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.341949+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:32:11.341949+0000","last_peered":"2024-05-08T23:32:11.341949+0000","last_clean":"2024-05-08T23:32:11.341949+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:32:11.341949+0000","last_undegraded":"2024-05-08T23:32:11.341949+0000","last_fullsized":"2024-05-08T23:32:11.341949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.679967+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:32:11.679967+0000","last_peered":"2024-05-08T23:32:11.679967+0000","last_clean":"2024-05-08T23:32:11.679967+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:32:11.679967+0000","last_undegraded":"2024-05-08T23:32:11.679967+0000","last_fullsized":"2024-05-08T23:32:11.679967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-05-08T23:32:11.680305+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:32:11.680305+0000","last_peered":"2024-05-08T23:32:11.680305+0000","last_clean":"2024-05-08T23:32:11.680305+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:32:11.680305+0000","last_undegraded":"2024-05-08T23:32:11.680305+0000","last_fullsized":"2024-05-08T23:32:11.680305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:32:32.523 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-05-08T23:32:32.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:32.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.699+0000 7f1446a16700 1 Processor -- start 2024-05-08T23:32:32.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.699+0000 7f1446a16700 1 -- start start 2024-05-08T23:32:32.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1446a16700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1440154970 0x7f144006ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1446a16700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f14400caad0 0x7f144015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1446a16700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144015b350 con 0x7f144006b280 2024-05-08T23:32:32.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1446a16700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144015e8a0 con 0x7f14400caad0 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1446a16700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144015a5d0 con 0x7f1440154970 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f143f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f144006b280 0x7f14400cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48608/0 (socket says 172.21.15.33:48608) 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f143f7fe700 1 -- 172.21.15.33:0/662304706 learned_addr learned my addr 172.21.15.33:0/662304706 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f1444fb3700 1 --2- 172.21.15.33:0/662304706 >> v2:172.21.15.42:3300/0 conn(0x7f14400caad0 0x7f144015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f143ffff700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1440154970 0x7f144006ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.700+0000 7f143effd700 1 -- 172.21.15.33:0/662304706 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216554178 0 0) 0x7f144015b350 con 0x7f144006b280 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f143effd700 1 -- 172.21.15.33:0/662304706 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c0035b0 con 0x7f144006b280 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f1444fb3700 1 -- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1440154970 msgr2=0x7f144006ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f1444fb3700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1440154970 0x7f144006ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f1444fb3700 1 -- 172.21.15.33:0/662304706 >> v1:172.21.15.33:6789/0 conn(0x7f144006b280 legacy=0x7f14400cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f1444fb3700 1 -- 172.21.15.33:0/662304706 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144015e5c0 con 0x7f14400caad0 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f1444fb3700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14400caad0 0x7f144015d9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1438009d90 tx=0x7f143800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed48e8bce6598fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f143effd700 1 -- 172.21.15.33:0/662304706 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800e070 con 0x7f14400caad0 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f143effd700 1 -- 172.21.15.33:0/662304706 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800bb20 con 0x7f14400caad0 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.701+0000 7f143effd700 1 -- 172.21.15.33:0/662304706 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438012430 con 0x7f14400caad0 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 -- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14400caad0 msgr2=0x7f144015d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14400caad0 0x7f144015d9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1438009d90 tx=0x7f143800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 -- 172.21.15.33:0/662304706 shutdown_connections 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14400caad0 0x7f144015d9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 --2- 172.21.15.33:0/662304706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1440154970 0x7f144006ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 -- 172.21.15.33:0/662304706 >> 172.21.15.33:0/662304706 conn(0x7f144006cd80 msgr2=0x7f144006d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 -- 172.21.15.33:0/662304706 shutdown_connections 2024-05-08T23:32:32.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.702+0000 7f1446a16700 1 -- 172.21.15.33:0/662304706 wait complete. 2024-05-08T23:32:32.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.703+0000 7f1446a16700 1 Processor -- start 2024-05-08T23:32:32.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.703+0000 7f1446a16700 1 -- start start 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f144006b280 0x7f14400c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1440154970 0x7f14401f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14401f3ea0 con 0x7f14400caad0 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14401f43d0 con 0x7f1440154970 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1446a16700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14401f0150 con 0x7f144006b280 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f143f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f143f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56816/0 (socket says 172.21.15.33:56816) 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f143f7fe700 1 -- 172.21.15.33:0/2030547942 learned_addr learned my addr 172.21.15.33:0/2030547942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f1444fb3700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1440154970 0x7f14401f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.704+0000 7f143ffff700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f144006b280 0x7f14400c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 -- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f144006b280 msgr2=0x7f14400c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f144006b280 0x7f14400c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 -- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1440154970 msgr2=0x7f14401f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1440154970 0x7f14401f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 -- 172.21.15.33:0/2030547942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438009a70 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143f7fe700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f142800c700 tx=0x7f142800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428005d70 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14280070d0 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14401e3780 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428007460 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.705+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14401e3a20 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.706+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144006c750 con 0x7f14400caad0 2024-05-08T23:32:32.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.707+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f14280075c0 con 0x7f14400caad0 2024-05-08T23:32:32.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.707+0000 7f143cff9700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 0x7f142006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:32.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.707+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1428088750 con 0x7f14400caad0 2024-05-08T23:32:32.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.707+0000 7f143ffff700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 0x7f142006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:32.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.708+0000 7f143ffff700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 0x7f142006d180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1430009b40 tx=0x7f1430009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:32.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.712+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1428053930 con 0x7f14400caad0 2024-05-08T23:32:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.931+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14401f0290 con 0x7f14400caad0 2024-05-08T23:32:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.932+0000 7f143cff9700 1 -- 172.21.15.33:0/2030547942 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f1428056f50 con 0x7f14400caad0 2024-05-08T23:32:32.932 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:32.933 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:11.332768+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.934+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 msgr2=0x7f142006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.934+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 0x7f142006d180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1430009b40 tx=0x7f1430009210 comp rx=0 tx=0).stop 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.934+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 msgr2=0x7f14401f05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f142800c700 tx=0x7f142800ca10 comp rx=0 tx=0).stop 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 shutdown_connections 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f142006acc0 0x7f142006d180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1440154970 0x7f14401f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f144006b280 0x7f14400c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 --2- 172.21.15.33:0/2030547942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400caad0 0x7f14401f05e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 >> 172.21.15.33:0/2030547942 conn(0x7f144006cd80 msgr2=0x7f14400ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 shutdown_connections 2024-05-08T23:32:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:32.935+0000 7f1446a16700 1 -- 172.21.15.33:0/2030547942 wait complete. 2024-05-08T23:32:32.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:08.545052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:09.301342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:32:32.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:33.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 Processor -- start 2024-05-08T23:32:33.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 -- start start 2024-05-08T23:32:33.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28b01546c0 0x7f28b0072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 0x7f28b015de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b006bb70 con 0x7f28b00c7420 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b00719b0 con 0x7f28b01546c0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.088+0000 7f28b5598700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b00c7860 con 0x7f28b015d9d0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28b01546c0 0x7f28b0072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f28b01546c0 0x7f28b0072ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56828/0 (socket says 172.21.15.33:56828) 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 -- 172.21.15.33:0/2952600704 learned_addr learned my addr 172.21.15.33:0/2952600704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28af7fe700 1 --2- 172.21.15.33:0/2952600704 >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 0x7f28b015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 -- 172.21.15.33:0/2952600704 >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 msgr2=0x7f28b015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 --2- 172.21.15.33:0/2952600704 >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 0x7f28b015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 -- 172.21.15.33:0/2952600704 >> v1:172.21.15.42:6789/0 conn(0x7f28b00c7420 legacy=0x7f28b006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 -- 172.21.15.33:0/2952600704 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0071c00 con 0x7f28b01546c0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28af7fe700 1 --2- 172.21.15.33:0/2952600704 >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 0x7f28b015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28aeffd700 1 --2- 172.21.15.33:0/2952600704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b01546c0 0x7f28b0072ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b0070c60 tx=0x7f28a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cd8eb0fb728c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28adffb700 1 -- 172.21.15.33:0/2952600704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000b870 con 0x7f28b01546c0 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28adffb700 1 -- 172.21.15.33:0/2952600704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a000b9d0 con 0x7f28b01546c0 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.089+0000 7f28adffb700 1 -- 172.21.15.33:0/2952600704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000bd60 con 0x7f28b01546c0 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 -- 172.21.15.33:0/2952600704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b01546c0 msgr2=0x7f28b0072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 --2- 172.21.15.33:0/2952600704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b01546c0 0x7f28b0072ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f28b0070c60 tx=0x7f28a0009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 -- 172.21.15.33:0/2952600704 shutdown_connections 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 --2- 172.21.15.33:0/2952600704 >> v2:172.21.15.42:3300/0 conn(0x7f28b015d9d0 0x7f28b015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 --2- 172.21.15.33:0/2952600704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b01546c0 0x7f28b0072ca0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 -- 172.21.15.33:0/2952600704 >> 172.21.15.33:0/2952600704 conn(0x7f28b00c9620 msgr2=0x7f28b00c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 -- 172.21.15.33:0/2952600704 shutdown_connections 2024-05-08T23:32:33.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.090+0000 7f28b5598700 1 -- 172.21.15.33:0/2952600704 wait complete. 2024-05-08T23:32:33.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 Processor -- start 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 -- start start 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00c7420 0x7f28b01f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f28b01546c0 0x7f28b00c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b015a050 con 0x7f28b015d9d0 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b015ae20 con 0x7f28b00c7420 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.092+0000 7f28b5598700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b015b100 con 0x7f28b01546c0 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56830/0 (socket says 172.21.15.33:56830) 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 -- 172.21.15.33:0/1986684590 learned_addr learned my addr 172.21.15.33:0/1986684590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28aeffd700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00c7420 0x7f28b01f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28ae7fc700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f28b01546c0 0x7f28b00c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 -- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f28b01546c0 msgr2=0x7f28b00c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f28b01546c0 0x7f28b00c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 -- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00c7420 msgr2=0x7f28b01f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00c7420 0x7f28b01f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 -- 172.21.15.33:0/1986684590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a0009550 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28af7fe700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009fa0 tx=0x7f28a800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8003a20 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a800f040 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b00c5930 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8003de0 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.093+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b00c5ef0 con 0x7f28b015d9d0 2024-05-08T23:32:33.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.094+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b00c6110 con 0x7f28b015d9d0 2024-05-08T23:32:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.095+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f28a8004030 con 0x7f28b015d9d0 2024-05-08T23:32:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.095+0000 7f28977fe700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 0x7f289c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.096+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28a8089590 con 0x7f28b015d9d0 2024-05-08T23:32:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.096+0000 7f28aeffd700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 0x7f289c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.096+0000 7f28aeffd700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 0x7f289c06d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a0014040 tx=0x7f28a0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.100+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f28a80546e0 con 0x7f28b015d9d0 2024-05-08T23:32:33.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.319+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28b0075460 con 0x7f28b015d9d0 2024-05-08T23:32:33.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.319+0000 7f28977fe700 1 -- 172.21.15.33:0/1986684590 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28a801a020 con 0x7f28b015d9d0 2024-05-08T23:32:33.320 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:32:33.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 msgr2=0x7f289c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 0x7f289c06d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a0014040 tx=0x7f28a0008030 comp rx=0 tx=0).stop 2024-05-08T23:32:33.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 msgr2=0x7f28b00c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009fa0 tx=0x7f28a800d460 comp rx=0 tx=0).stop 2024-05-08T23:32:33.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 shutdown_connections 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f289c06ac90 0x7f289c06d150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b015d9d0 0x7f28b00c3170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28b00c7420 0x7f28b01f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 --2- 172.21.15.33:0/1986684590 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f28b01546c0 0x7f28b00c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 >> 172.21.15.33:0/1986684590 conn(0x7f28b00c9620 msgr2=0x7f28b00c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 shutdown_connections 2024-05-08T23:32:33.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.322+0000 7f28b5598700 1 -- 172.21.15.33:0/1986684590 wait complete. 2024-05-08T23:32:33.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.475+0000 7f07278c1700 1 Processor -- start 2024-05-08T23:32:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.475+0000 7f07278c1700 1 -- start start 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f07278c1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f072006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f07278c1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0720158bf0 con 0x7f0720152360 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f07278c1700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072006b5e0 con 0x7f07200c90d0 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f07278c1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0720152740 con 0x7f07200b2de0 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f072565d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0720152360 0x7f07200c89c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48620/0 (socket says 172.21.15.33:48620) 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f072565d700 1 -- 172.21.15.33:0/83802811 learned_addr learned my addr 172.21.15.33:0/83802811 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f0724e5c700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f072006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648493524 0 0) 0x7f0720158bf0 con 0x7f0720152360 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.476+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f070c0035b0 con 0x7f0720152360 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290717463 0 0) 0x7f072006b5e0 con 0x7f07200c90d0 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0720158bf0 con 0x7f07200c90d0 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756178206 0 0) 0x7f070c0035b0 con 0x7f0720152360 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 msgr2=0x7f072006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f072006a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 >> v1:172.21.15.42:6789/0 conn(0x7f07200c90d0 legacy=0x7f072006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720159130 con 0x7f0720152360 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f071c001670 con 0x7f0720152360 2024-05-08T23:32:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 >> v1:172.21.15.33:6789/0 conn(0x7f0720152360 legacy=0x7f07200c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f070c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f070c004120 0x7f070c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070c004d90 0x7f070c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720152740 con 0x7f070c004d90 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c00baf0 con 0x7f070c004120 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c00bcb0 con 0x7f07200b2de0 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070c004d90 0x7f070c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f072565d700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f070c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0724e5c700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f070c004120 0x7f070c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 -- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 msgr2=0x7f070c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f07200b2de0 0x7f070c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 -- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f070c004120 msgr2=0x7f070c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f070c004120 0x7f070c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 -- 172.21.15.33:0/83802811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0035b0 con 0x7f070c004d90 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.477+0000 7f0725e5e700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070c004d90 0x7f070c005210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f071800c2f0 tx=0x7f070c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167e03407c174ade server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718001f30 con 0x7f070c004d90 2024-05-08T23:32:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071800ce90 con 0x7f070c004d90 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f0717fff700 1 -- 172.21.15.33:0/83802811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718003c40 con 0x7f070c004d90 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 -- 172.21.15.33:0/83802811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070c004d90 msgr2=0x7f070c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 --2- 172.21.15.33:0/83802811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f070c004d90 0x7f070c005210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f071800c2f0 tx=0x7f070c005880 comp rx=0 tx=0).stop 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f0725e5e700 1 -- 172.21.15.33:0/83802811 reap_dead start 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 -- 172.21.15.33:0/83802811 shutdown_connections 2024-05-08T23:32:33.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 -- 172.21.15.33:0/83802811 >> 172.21.15.33:0/83802811 conn(0x7f072006ce60 msgr2=0x7f072006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 -- 172.21.15.33:0/83802811 shutdown_connections 2024-05-08T23:32:33.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.478+0000 7f07278c1700 1 -- 172.21.15.33:0/83802811 wait complete. 2024-05-08T23:32:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 Processor -- start 2024-05-08T23:32:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 -- start start 2024-05-08T23:32:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 0x7f07201ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0720151230 0x7f07201e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07200c8440 con 0x7f07201586a0 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07201434e0 con 0x7f0720150e50 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.480+0000 7f07278c1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07201eec70 con 0x7f0720151230 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56848/0 (socket says 172.21.15.33:56848) 2024-05-08T23:32:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 -- 172.21.15.33:0/3162969704 learned_addr learned my addr 172.21.15.33:0/3162969704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f072565d700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 0x7f07201ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0724e5c700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0720151230 0x7f07201e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 -- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0720151230 msgr2=0x7f07201e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0720151230 0x7f07201e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 -- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 msgr2=0x7f07201ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 0x7f07201ee730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 -- 172.21.15.33:0/3162969704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720155f30 con 0x7f07201586a0 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f072565d700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 0x7f07201ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0725e5e700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0718005040 tx=0x7f0718001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180050d0 con 0x7f07201586a0 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718005250 con 0x7f07201586a0 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07201580f0 con 0x7f07201586a0 2024-05-08T23:32:33.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.481+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718005600 con 0x7f07201586a0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.482+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07201564b0 con 0x7f07201586a0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.483+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0720073770 con 0x7f07201586a0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.483+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0718019040 con 0x7f07201586a0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.484+0000 7f0715ffb700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 0x7f070406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.484+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07180897d0 con 0x7f07201586a0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.484+0000 7f072565d700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 0x7f070406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.484+0000 7f072565d700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 0x7f070406d150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f071c001eb0 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.488+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0718054a40 con 0x7f07201586a0 2024-05-08T23:32:33.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.702+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0720075460 con 0x7f07201586a0 2024-05-08T23:32:33.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.703+0000 7f0715ffb700 1 -- 172.21.15.33:0/3162969704 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07180011c0 con 0x7f07201586a0 2024-05-08T23:32:33.704 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T23:32:33.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 msgr2=0x7f070406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 0x7f070406d150 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f071c001eb0 tx=0x7f071c009040 comp rx=0 tx=0).stop 2024-05-08T23:32:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 msgr2=0x7f07200c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0718005040 tx=0x7f0718001650 comp rx=0 tx=0).stop 2024-05-08T23:32:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 shutdown_connections 2024-05-08T23:32:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f070406ac90 0x7f070406d150 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07201586a0 0x7f07200c62a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0720150e50 0x7f07201ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 --2- 172.21.15.33:0/3162969704 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0720151230 0x7f07201e2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.705+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 >> 172.21.15.33:0/3162969704 conn(0x7f072006ce60 msgr2=0x7f072006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.706+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 shutdown_connections 2024-05-08T23:32:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.706+0000 7f07278c1700 1 -- 172.21.15.33:0/3162969704 wait complete. 2024-05-08T23:32:33.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:33.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.859+0000 7fdda86eb700 1 Processor -- start 2024-05-08T23:32:33.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.859+0000 7fdda86eb700 1 -- start start 2024-05-08T23:32:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.860+0000 7fdda86eb700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdda00c9280 0x7fdda00c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.860+0000 7fdda86eb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdda0157a90 0x7fdda0150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.860+0000 7fdda86eb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda00c8610 con 0x7fdda00c8ea0 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.860+0000 7fdda86eb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda00c4470 con 0x7fdda0157a90 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.860+0000 7fdda86eb700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda00c7d90 con 0x7fdda00c9280 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdda0157a90 0x7fdda0150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6487700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdda00c8ea0 0x7fdda0071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48636/0 (socket says 172.21.15.33:48636) 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6487700 1 -- 172.21.15.33:0/2769700089 learned_addr learned my addr 172.21.15.33:0/2769700089 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda5c86700 1 --2- 172.21.15.33:0/2769700089 >> v2:172.21.15.42:3300/0 conn(0x7fdda00c9280 0x7fdda00c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 -- 172.21.15.33:0/2769700089 >> v2:172.21.15.42:3300/0 conn(0x7fdda00c9280 msgr2=0x7fdda00c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 --2- 172.21.15.33:0/2769700089 >> v2:172.21.15.42:3300/0 conn(0x7fdda00c9280 0x7fdda00c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 -- 172.21.15.33:0/2769700089 >> v1:172.21.15.33:6789/0 conn(0x7fdda00c8ea0 legacy=0x7fdda0071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 -- 172.21.15.33:0/2769700089 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0157f20 con 0x7fdda0157a90 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda5485700 1 -- 172.21.15.33:0/2769700089 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659140014 0 0) 0x7fdda00c8610 con 0x7fdda00c8ea0 2024-05-08T23:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda6c88700 1 --2- 172.21.15.33:0/2769700089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda0150d00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdda006f2f0 tx=0x7fdd9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7422512921753038 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda5485700 1 -- 172.21.15.33:0/2769700089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800e070 con 0x7fdda0157a90 2024-05-08T23:32:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda5485700 1 -- 172.21.15.33:0/2769700089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9800bb20 con 0x7fdda0157a90 2024-05-08T23:32:33.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.861+0000 7fdda5485700 1 -- 172.21.15.33:0/2769700089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98012430 con 0x7fdda0157a90 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 -- 172.21.15.33:0/2769700089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 msgr2=0x7fdda0150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 --2- 172.21.15.33:0/2769700089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda0150d00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdda006f2f0 tx=0x7fdd9800b330 comp rx=0 tx=0).stop 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 -- 172.21.15.33:0/2769700089 shutdown_connections 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 --2- 172.21.15.33:0/2769700089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda0150d00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 --2- 172.21.15.33:0/2769700089 >> v2:172.21.15.42:3300/0 conn(0x7fdda00c9280 0x7fdda00c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 -- 172.21.15.33:0/2769700089 >> 172.21.15.33:0/2769700089 conn(0x7fdda006a990 msgr2=0x7fdda006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 -- 172.21.15.33:0/2769700089 shutdown_connections 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.862+0000 7fdda86eb700 1 -- 172.21.15.33:0/2769700089 wait complete. 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.863+0000 7fdda86eb700 1 Processor -- start 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.863+0000 7fdda86eb700 1 -- start start 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 0x7fdda01ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdda00c9280 0x7fdda01ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01f1980 con 0x7fdda0157a90 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda0155f60 con 0x7fdda00c8ea0 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda86eb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda0156240 con 0x7fdda00c9280 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda6c88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda6c88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56870/0 (socket says 172.21.15.33:56870) 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda6c88700 1 -- 172.21.15.33:0/3958284706 learned_addr learned my addr 172.21.15.33:0/3958284706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.864+0000 7fdda6487700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 0x7fdda01ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda5c86700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdda00c9280 0x7fdda01ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 -- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdda00c9280 msgr2=0x7fdda01ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdda00c9280 0x7fdda01ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 -- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 msgr2=0x7fdda01ee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 0x7fdda01ee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 -- 172.21.15.33:0/3958284706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98009a70 con 0x7fdda0157a90 2024-05-08T23:32:33.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6487700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 0x7fdda01ee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda6c88700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdda006b550 tx=0x7fdd98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98012430 con 0x7fdda0157a90 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98020070 con 0x7fdda0157a90 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004280 con 0x7fdda0157a90 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda0156520 con 0x7fdda0157a90 2024-05-08T23:32:33.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.865+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01edaf0 con 0x7fdda0157a90 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.866+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda0066dc0 con 0x7fdda0157a90 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.867+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdd980043e0 con 0x7fdda0157a90 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.867+0000 7fdd92ffd700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 0x7fdd8406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.867+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd98022070 con 0x7fdda0157a90 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.867+0000 7fdda6487700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 0x7fdd8406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:33.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.868+0000 7fdda6487700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 0x7fdd8406d150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd94000f60 tx=0x7fdd94007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:33.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:33.872+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdd98056300 con 0x7fdda0157a90 2024-05-08T23:32:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.089+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdda00c68e0 con 0x7fdda0157a90 2024-05-08T23:32:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.089+0000 7fdd92ffd700 1 -- 172.21.15.33:0/3958284706 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd98059920 con 0x7fdda0157a90 2024-05-08T23:32:34.090 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T23:32:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 msgr2=0x7fdd8406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 0x7fdd8406d150 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd94000f60 tx=0x7fdd94007ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 msgr2=0x7fdda01f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdda006b550 tx=0x7fdd98004ad0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 shutdown_connections 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdd8406ac90 0x7fdd8406d150 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdda0157a90 0x7fdda01f1440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdda00c8ea0 0x7fdda01ee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 --2- 172.21.15.33:0/3958284706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdda00c9280 0x7fdda01ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.091+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 >> 172.21.15.33:0/3958284706 conn(0x7fdda006a990 msgr2=0x7fdda006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.092+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 shutdown_connections 2024-05-08T23:32:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.092+0000 7fdda86eb700 1 -- 172.21.15.33:0/3958284706 wait complete. 2024-05-08T23:32:34.105 DEBUG:tasks.mon_thrash:Yielding 2024-05-08T23:32:34.106 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T23:32:34.117 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:32:34.118 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T23:32:34.118 INFO:tasks.workunit:timeout=3h 2024-05-08T23:32:34.118 INFO:tasks.workunit:cleanup=True 2024-05-08T23:32:34.118 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:34.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 2fh/47d Inode: 1 Links: 2 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:cephfs_t:s0 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 23:32:15.334592562 +0000 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 23:32:29.175815680 +0000 2024-05-08T23:32:34.139 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-08 23:32:15.334592562 +0000 2024-05-08T23:32:34.140 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:34.140 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T23:32:34.213 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:32:34.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.258+0000 7f4bd4316700 1 Processor -- start 2024-05-08T23:32:34.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.258+0000 7f4bd4316700 1 -- start start 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd4316700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4bcc0c8ea0 0x7f4bcc157970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd4316700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bcc0668c0 0x7f4bcc150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd4316700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bcc070a90 con 0x7f4bcc0c9280 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd4316700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc0c4390 con 0x7f4bcc0668c0 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd4316700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bcc157eb0 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd28b3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bcc0668c0 0x7f4bcc150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd28b3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4bcc0668c0 0x7f4bcc150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56880/0 (socket says 172.21.15.33:56880) 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd28b3700 1 -- 172.21.15.33:0/2750710022 learned_addr learned my addr 172.21.15.33:0/2750710022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd20b2700 1 --2- 172.21.15.33:0/2750710022 >> v2:172.21.15.42:3300/0 conn(0x7f4bcc0c8ea0 0x7f4bcc157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd28b3700 1 -- 172.21.15.33:0/2750710022 >> v2:172.21.15.42:3300/0 conn(0x7f4bcc0c8ea0 msgr2=0x7f4bcc157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.259+0000 7f4bd28b3700 1 --2- 172.21.15.33:0/2750710022 >> v2:172.21.15.42:3300/0 conn(0x7f4bcc0c8ea0 0x7f4bcc157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd28b3700 1 -- 172.21.15.33:0/2750710022 >> v1:172.21.15.42:6789/0 conn(0x7f4bcc0c9280 legacy=0x7f4bcc070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd28b3700 1 -- 172.21.15.33:0/2750710022 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0c6f70 con 0x7f4bcc0668c0 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd28b3700 1 --2- 172.21.15.33:0/2750710022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0668c0 0x7f4bcc150ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0c9ca0 tx=0x7f4bc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cd65a97265db02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd10b0700 1 -- 172.21.15.33:0/2750710022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400e070 con 0x7f4bcc0668c0 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd10b0700 1 -- 172.21.15.33:0/2750710022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc400b910 con 0x7f4bcc0668c0 2024-05-08T23:32:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.260+0000 7f4bd10b0700 1 -- 172.21.15.33:0/2750710022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400bcc0 con 0x7f4bcc0668c0 2024-05-08T23:32:34.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 -- 172.21.15.33:0/2750710022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0668c0 msgr2=0x7f4bcc150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 --2- 172.21.15.33:0/2750710022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0668c0 0x7f4bcc150ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0c9ca0 tx=0x7f4bc400b330 comp rx=0 tx=0).stop 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 -- 172.21.15.33:0/2750710022 shutdown_connections 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 --2- 172.21.15.33:0/2750710022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0668c0 0x7f4bcc150ce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 --2- 172.21.15.33:0/2750710022 >> v2:172.21.15.42:3300/0 conn(0x7f4bcc0c8ea0 0x7f4bcc157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 -- 172.21.15.33:0/2750710022 >> 172.21.15.33:0/2750710022 conn(0x7f4bcc06a990 msgr2=0x7f4bcc06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 -- 172.21.15.33:0/2750710022 shutdown_connections 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.261+0000 7f4bd4316700 1 -- 172.21.15.33:0/2750710022 wait complete. 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.262+0000 7f4bd4316700 1 Processor -- start 2024-05-08T23:32:34.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.262+0000 7f4bd4316700 1 -- start start 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4bcc0668c0 0x7f4bcc1eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bcc0c9280 0x7f4bcc0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc14a260 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1efa70 con 0x7f4bcc0c9280 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd4316700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0cae70 con 0x7f4bcc0668c0 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56882/0 (socket says 172.21.15.33:56882) 2024-05-08T23:32:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 -- 172.21.15.33:0/1599028158 learned_addr learned my addr 172.21.15.33:0/1599028158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd20b2700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4bcc0668c0 0x7f4bcc1eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd28b3700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bcc0c9280 0x7f4bcc0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 -- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4bcc0668c0 msgr2=0x7f4bcc1eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4bcc0668c0 0x7f4bcc1eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 -- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bcc0c9280 msgr2=0x7f4bcc0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bcc0c9280 0x7f4bcc0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.263+0000 7f4bd18b1700 1 -- 172.21.15.33:0/1599028158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4009b30 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bd18b1700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c450 tx=0x7f4bbc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0052d0 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc005430 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc149e70 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0038c0 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.264+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc14a4b0 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.265+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc1efbe0 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.266+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4bbc003a20 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.267+0000 7f4bc2ffd700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 0x7f4bb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.267+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bbc088b60 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.267+0000 7f4bd20b2700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 0x7f4bb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.267+0000 7f4bd20b2700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 0x7f4bb006d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009b40 tx=0x7f4bc8009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.271+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4bbc053c90 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.273 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T23:32:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.548+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bcc075460 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.549+0000 7f4bc2ffd700 1 -- 172.21.15.33:0/1599028158 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7f4bbc017020 con 0x7f4bcc0c8ea0 2024-05-08T23:32:34.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:32:34.549 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 msgr2=0x7f4bb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 0x7f4bb006d150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009b40 tx=0x7f4bc8009210 comp rx=0 tx=0).stop 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 msgr2=0x7f4bcc1f0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00c450 tx=0x7f4bbc00c760 comp rx=0 tx=0).stop 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 shutdown_connections 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4bb006ac90 0x7f4bb006d150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bcc0c9280 0x7f4bcc0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4bcc0668c0 0x7f4bcc1eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 --2- 172.21.15.33:0/1599028158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bcc0c8ea0 0x7f4bcc1f0440 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.551+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 >> 172.21.15.33:0/1599028158 conn(0x7f4bcc06a990 msgr2=0x7f4bcc071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.552+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 shutdown_connections 2024-05-08T23:32:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.552+0000 7f4bd4316700 1 -- 172.21.15.33:0/1599028158 wait complete. 2024-05-08T23:32:34.553 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:32:34.565 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-05-08T23:32:34.566 INFO:tasks.mon_thrash.mon_thrasher:seed: 1715211152, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-05-08T23:32:34.566 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:32:34.566 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:32:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.758+0000 7f43ee821700 1 Processor -- start 2024-05-08T23:32:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.758+0000 7f43ee821700 1 -- start start 2024-05-08T23:32:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43ee821700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43ee821700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43e80c38a0 0x7f43e80c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43ee821700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e8070450 con 0x7f43e80c9a60 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43ee821700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80c2e90 con 0x7f43e80c38a0 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43ee821700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8070730 con 0x7f43e8152350 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43e80c38a0 0x7f43e80c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43e80c38a0 0x7f43e80c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56894/0 (socket says 172.21.15.33:56894) 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 -- 172.21.15.33:0/686979912 learned_addr learned my addr 172.21.15.33:0/686979912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e7fff700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 -- 172.21.15.33:0/686979912 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 msgr2=0x7f43e8158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 -- 172.21.15.33:0/686979912 >> v1:172.21.15.42:6789/0 conn(0x7f43e80c9a60 legacy=0x7f43e80c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e77fe700 1 -- 172.21.15.33:0/686979912 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8070a10 con 0x7f43e80c38a0 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.759+0000 7f43e6ffd700 1 -- 172.21.15.33:0/686979912 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695666938 0 0) 0x7f43e8070450 con 0x7f43e80c9a60 2024-05-08T23:32:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43e77fe700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e80c9520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0098f0 tx=0x7f43dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76698399b8d99a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43e6ffd700 1 -- 172.21.15.33:0/686979912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc011830 con 0x7f43e80c38a0 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43e6ffd700 1 -- 172.21.15.33:0/686979912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc011990 con 0x7f43e80c38a0 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43e6ffd700 1 -- 172.21.15.33:0/686979912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc011d20 con 0x7f43e80c38a0 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 -- 172.21.15.33:0/686979912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 msgr2=0x7f43e80c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e80c9520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0098f0 tx=0x7f43dc009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 -- 172.21.15.33:0/686979912 shutdown_connections 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 --2- 172.21.15.33:0/686979912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e80c9520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.760+0000 7f43ee821700 1 -- 172.21.15.33:0/686979912 >> 172.21.15.33:0/686979912 conn(0x7f43e80666c0 msgr2=0x7f43e8066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.761+0000 7f43ee821700 1 -- 172.21.15.33:0/686979912 shutdown_connections 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.761+0000 7f43ee821700 1 -- 172.21.15.33:0/686979912 wait complete. 2024-05-08T23:32:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.762+0000 7f43ee821700 1 Processor -- start 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 -- start start 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e80c9a60 0x7f43e8150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8149810 con 0x7f43e80c38a0 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e814a8d0 con 0x7f43e80c9a60 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.763+0000 7f43ee821700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81f1870 con 0x7f43e8152350 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56900/0 (socket says 172.21.15.33:56900) 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 -- 172.21.15.33:0/1987295081 learned_addr learned my addr 172.21.15.33:0/1987295081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43ecdbe700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e77fe700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e80c9a60 0x7f43e8150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 -- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 msgr2=0x7f43e8148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 -- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e80c9a60 msgr2=0x7f43e8150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e80c9a60 0x7f43e8150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 -- 172.21.15.33:0/1987295081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc009550 con 0x7f43e80c38a0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43ecdbe700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e7fff700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e870 tx=0x7f43d800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8009590 con 0x7f43e80c38a0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.764+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8013680 con 0x7f43e80c38a0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.765+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e81f1bb0 con 0x7f43e80c38a0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.765+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80139f0 con 0x7f43e80c38a0 2024-05-08T23:32:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.765+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81518c0 con 0x7f43e80c38a0 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.766+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f43d8013b50 con 0x7f43e80c38a0 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.766+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8075460 con 0x7f43e80c38a0 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.766+0000 7f43e4ff9700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 0x7f43d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.767+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43d80891c0 con 0x7f43e80c38a0 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.767+0000 7f43e77fe700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 0x7f43d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:34.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.767+0000 7f43e77fe700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 0x7f43d406d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009c40 tx=0x7f43dc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:34.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:34.771+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f43d8054430 con 0x7f43e80c38a0 2024-05-08T23:32:35.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.107+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f43e814b320 con 0x7f43e80c38a0 2024-05-08T23:32:35.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.107+0000 7f43e4ff9700 1 -- 172.21.15.33:0/1987295081 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f43d8057a50 con 0x7f43e80c38a0 2024-05-08T23:32:35.109 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":42,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 msgr2=0x7f43d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 0x7f43d406d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009c40 tx=0x7f43dc008030 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 msgr2=0x7f43e814a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e870 tx=0x7f43d800eb80 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 shutdown_connections 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43d406ac90 0x7f43d406d150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43e8152350 0x7f43e8148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e80c38a0 0x7f43e814a390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 --2- 172.21.15.33:0/1987295081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43e80c9a60 0x7f43e8150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.109+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 >> 172.21.15.33:0/1987295081 conn(0x7f43e80666c0 msgr2=0x7f43e8155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.110+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 shutdown_connections 2024-05-08T23:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.110+0000 7f43ee821700 1 -- 172.21.15.33:0/1987295081 wait complete. 2024-05-08T23:32:35.124 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:32:35.124 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:32:35.124 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:32:35.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:32:35.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.262+0000 7f3c56ffe700 1 Processor -- start 2024-05-08T23:32:35.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.262+0000 7f3c56ffe700 1 -- start start 2024-05-08T23:32:35.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c56ffe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c50159520 0x7f3c500ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c56ffe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3c500c91b0 0x7f3c500ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c56ffe700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5006e3a0 con 0x7f3c50152260 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c56ffe700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c50158620 con 0x7f3c50159520 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c56ffe700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5006f280 con 0x7f3c500c91b0 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.263+0000 7f3c4ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c50159520 0x7f3c500ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c50159520 0x7f3c500ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56914/0 (socket says 172.21.15.33:56914) 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 -- 172.21.15.33:0/768428271 learned_addr learned my addr 172.21.15.33:0/768428271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c5559b700 1 --2- 172.21.15.33:0/768428271 >> v2:172.21.15.42:3300/0 conn(0x7f3c500c91b0 0x7f3c500ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 -- 172.21.15.33:0/768428271 >> v2:172.21.15.42:3300/0 conn(0x7f3c500c91b0 msgr2=0x7f3c500ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/768428271 >> v2:172.21.15.42:3300/0 conn(0x7f3c500c91b0 0x7f3c500ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 -- 172.21.15.33:0/768428271 >> v1:172.21.15.42:6789/0 conn(0x7f3c50152260 legacy=0x7f3c50158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 -- 172.21.15.33:0/768428271 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5006ecc0 con 0x7f3c50159520 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/768428271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50159520 0x7f3c500ce340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0098f0 tx=0x7f3c3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7515490d3c8ac6bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4f7fe700 1 -- 172.21.15.33:0/768428271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c011830 con 0x7f3c50159520 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4f7fe700 1 -- 172.21.15.33:0/768428271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c011990 con 0x7f3c50159520 2024-05-08T23:32:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.264+0000 7f3c4f7fe700 1 -- 172.21.15.33:0/768428271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c011d20 con 0x7f3c50159520 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 -- 172.21.15.33:0/768428271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50159520 msgr2=0x7f3c500ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/768428271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50159520 0x7f3c500ce340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0098f0 tx=0x7f3c3c009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 -- 172.21.15.33:0/768428271 shutdown_connections 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/768428271 >> v2:172.21.15.42:3300/0 conn(0x7f3c500c91b0 0x7f3c500ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/768428271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50159520 0x7f3c500ce340 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 -- 172.21.15.33:0/768428271 >> 172.21.15.33:0/768428271 conn(0x7f3c50072670 msgr2=0x7f3c50072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 -- 172.21.15.33:0/768428271 shutdown_connections 2024-05-08T23:32:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.265+0000 7f3c56ffe700 1 -- 172.21.15.33:0/768428271 wait complete. 2024-05-08T23:32:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.266+0000 7f3c56ffe700 1 Processor -- start 2024-05-08T23:32:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.266+0000 7f3c56ffe700 1 -- start start 2024-05-08T23:32:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 0x7f3c50149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c50152260 0x7f3c50142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c50159520 0x7f3c50142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c500c5a70 con 0x7f3c500c91b0 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50151470 con 0x7f3c50152260 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c56ffe700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c501f1450 con 0x7f3c50159520 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c54d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 0x7f3c50149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c54d9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 0x7f3c50149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56926/0 (socket says 172.21.15.33:56926) 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c54d9a700 1 -- 172.21.15.33:0/3300944334 learned_addr learned my addr 172.21.15.33:0/3300944334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.267+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c50152260 0x7f3c50142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c5559b700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c50159520 0x7f3c50142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c50159520 msgr2=0x7f3c50142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c50159520 0x7f3c50142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c50152260 msgr2=0x7f3c50142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c50152260 0x7f3c50142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c009550 con 0x7f3c500c91b0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c54d9a700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 0x7f3c50149160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c4e0 tx=0x7f3c4400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440052d0 con 0x7f3c500c91b0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c44005430 con 0x7f3c500c91b0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440038c0 con 0x7f3c500c91b0 2024-05-08T23:32:35.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c501f1730 con 0x7f3c500c91b0 2024-05-08T23:32:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.268+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c501f1cf0 con 0x7f3c500c91b0 2024-05-08T23:32:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.269+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 0x7f3c501edd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.269+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5006b420 con 0x7f3c500c35d0 2024-05-08T23:32:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.269+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 0x7f3c501edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.269+0000 7f3c4ffff700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c501f15c0 con 0x7f3c500c35d0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 0x7f3c501edd40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0098f0 tx=0x7f3c3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c01f070 con 0x7f3c500c35d0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c44003a20 con 0x7f3c500c91b0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c4d7fa700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c3406acd0 0x7f3c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c44088750 con 0x7f3c500c91b0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.270+0000 7f3c5559b700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c3406acd0 0x7f3c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3c3c0041a0 con 0x7f3c500c35d0 2024-05-08T23:32:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 msgr2=0x7f3c501edd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c4d7fa700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 0x7f3c501edd40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0098f0 tx=0x7f3c3c004750 comp rx=0 tx=0).stop 2024-05-08T23:32:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 msgr2=0x7f3c501edd40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c4d7fa700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c35d0 0x7f3c501edd40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.271+0000 7f3c5559b700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c3406acd0 0x7f3c3406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c500c6560 tx=0x7f3c4800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 0x7f3c501ee280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c500c2e90 con 0x7f3c50143360 2024-05-08T23:32:35.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 0x7f3c501ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c4ffff700 1 -- 172.21.15.33:0/3300944334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3c50158240 con 0x7f3c50143360 2024-05-08T23:32:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c4ffff700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 0x7f3c501ee280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009130 tx=0x7f3c3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.279+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c02b830 con 0x7f3c50143360 2024-05-08T23:32:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4969 (secure 0 0 0) 0x7f3c3c02f070 con 0x7f3c50143360 2024-05-08T23:32:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 msgr2=0x7f3c501ee280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c4d7fa700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 0x7f3c501ee280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009130 tx=0x7f3c3c011000 comp rx=0 tx=0).stop 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c4d7fa700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 msgr2=0x7f3c501ee280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c4d7fa700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c50143360 0x7f3c501ee280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 4, 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:32:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 43, 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.283 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:32:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.285 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.286 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:32:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:32:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:32:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:32:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:32:35.294 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c3406acd0 msgr2=0x7f3c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c3406acd0 0x7f3c3406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c500c6560 tx=0x7f3c4800b040 comp rx=0 tx=0).stop 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 msgr2=0x7f3c50149160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 --2- 172.21.15.33:0/3300944334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c500c91b0 0x7f3c50149160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c4e0 tx=0x7f3c4400c7f0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c5559b700 1 -- 172.21.15.33:0/3300944334 reap_dead start 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c5559b700 1 -- 172.21.15.33:0/3300944334 reap_dead start 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 shutdown_connections 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 >> 172.21.15.33:0/3300944334 conn(0x7f3c50072670 msgr2=0x7f3c5014b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 shutdown_connections 2024-05-08T23:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.280+0000 7f3c56ffe700 1 -- 172.21.15.33:0/3300944334 wait complete. 2024-05-08T23:32:35.296 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:32:35.297 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:32:35.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:32:35.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.434+0000 7fa763c47700 1 Processor -- start 2024-05-08T23:32:35.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.434+0000 7fa763c47700 1 -- start start 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa763c47700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa75c0cb980 0x7fa75c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa763c47700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa75c15dd90 0x7fa75c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa763c47700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c0cc270 con 0x7fa75c154480 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa763c47700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c0ccb90 con 0x7fa75c0cb980 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa763c47700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c06e560 con 0x7fa75c15dd90 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7611e2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa75c0cb980 0x7fa75c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7619e3700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa75c154480 0x7fa75c0cb200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47092/0 (socket says 172.21.15.33:47092) 2024-05-08T23:32:35.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7619e3700 1 -- 172.21.15.33:0/3998938470 learned_addr learned my addr 172.21.15.33:0/3998938470 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:35.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7621e4700 1 --2- 172.21.15.33:0/3998938470 >> v2:172.21.15.42:3300/0 conn(0x7fa75c15dd90 0x7fa75c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7609e1700 1 -- 172.21.15.33:0/3998938470 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378724854 0 0) 0x7fa75c0cc270 con 0x7fa75c154480 2024-05-08T23:32:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7609e1700 1 -- 172.21.15.33:0/3998938470 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7480035b0 con 0x7fa75c154480 2024-05-08T23:32:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.435+0000 7fa7611e2700 1 -- 172.21.15.33:0/3998938470 >> v2:172.21.15.42:3300/0 conn(0x7fa75c15dd90 msgr2=0x7fa75c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7611e2700 1 --2- 172.21.15.33:0/3998938470 >> v2:172.21.15.42:3300/0 conn(0x7fa75c15dd90 0x7fa75c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7611e2700 1 -- 172.21.15.33:0/3998938470 >> v1:172.21.15.33:6789/0 conn(0x7fa75c154480 legacy=0x7fa75c0cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7611e2700 1 -- 172.21.15.33:0/3998938470 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0cc420 con 0x7fa75c0cb980 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7611e2700 1 --2- 172.21.15.33:0/3998938470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c15d850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa75c15e910 tx=0x7fa74c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13df4acd01fb1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7609e1700 1 -- 172.21.15.33:0/3998938470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c011830 con 0x7fa75c0cb980 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7609e1700 1 -- 172.21.15.33:0/3998938470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c011990 con 0x7fa75c0cb980 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.436+0000 7fa7609e1700 1 -- 172.21.15.33:0/3998938470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c011d20 con 0x7fa75c0cb980 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 -- 172.21.15.33:0/3998938470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 msgr2=0x7fa75c15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 --2- 172.21.15.33:0/3998938470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c15d850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa75c15e910 tx=0x7fa74c009b20 comp rx=0 tx=0).stop 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 -- 172.21.15.33:0/3998938470 shutdown_connections 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 --2- 172.21.15.33:0/3998938470 >> v2:172.21.15.42:3300/0 conn(0x7fa75c15dd90 0x7fa75c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 --2- 172.21.15.33:0/3998938470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c15d850 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 -- 172.21.15.33:0/3998938470 >> 172.21.15.33:0/3998938470 conn(0x7fa75c06eb10 msgr2=0x7fa75c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 -- 172.21.15.33:0/3998938470 shutdown_connections 2024-05-08T23:32:35.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.437+0000 7fa763c47700 1 -- 172.21.15.33:0/3998938470 wait complete. 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.438+0000 7fa763c47700 1 Processor -- start 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.438+0000 7fa763c47700 1 -- start start 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c0c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c154480 0x7fa75c0c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa75c15dd90 0x7fa75c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0c5260 con 0x7fa75c0cb980 2024-05-08T23:32:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c154be0 con 0x7fa75c154480 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa763c47700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1578f0 con 0x7fa75c15dd90 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c0c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c0c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49742/0 (socket says 172.21.15.33:49742) 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 -- 172.21.15.33:0/2233473040 learned_addr learned my addr 172.21.15.33:0/2233473040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7621e4700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa75c15dd90 0x7fa75c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7611e2700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c154480 0x7fa75c0c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa75c15dd90 msgr2=0x7fa75c155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa75c15dd90 0x7fa75c155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c154480 msgr2=0x7fa75c0c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c154480 0x7fa75c0c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0094a0 con 0x7fa75c0cb980 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7619e3700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c0c57b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa758001eb0 tx=0x7fa75800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.439+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758005d70 con 0x7fa75c0cb980 2024-05-08T23:32:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580070d0 con 0x7fa75c0cb980 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c154e30 con 0x7fa75c0cb980 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758007460 con 0x7fa75c0cb980 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1553f0 con 0x7fa75c0cb980 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa763c47700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 0x7fa75c156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.440+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c15b2c0 con 0x7fa75c157390 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.441+0000 7fa7611e2700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 0x7fa75c156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.441+0000 7fa7611e2700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa75c06fab0 con 0x7fa75c157390 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.441+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa7580075c0 con 0x7fa75c0cb980 2024-05-08T23:32:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.441+0000 7fa7611e2700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 0x7fa75c156130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa75c15e910 tx=0x7fa74c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.441+0000 7fa7527fc700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa74006ac90 0x7fa74006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa758088680 con 0x7fa75c0cb980 2024-05-08T23:32:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7621e4700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa74006ac90 0x7fa74006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c028070 con 0x7fa75c157390 2024-05-08T23:32:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa74c004510 con 0x7fa75c157390 2024-05-08T23:32:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 msgr2=0x7fa75c156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 0x7fa75c156130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa75c15e910 tx=0x7fa74c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 msgr2=0x7fa75c156130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7527fc700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c157390 0x7fa75c156130 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa75c15e910 tx=0x7fa74c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.442+0000 7fa7621e4700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa74006ac90 0x7fa74006d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa754011340 tx=0x7fa75400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.450+0000 7fa763c47700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 0x7fa75c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.450+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c074b30 con 0x7fa75c0c4b40 2024-05-08T23:32:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.450+0000 7fa7611e2700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 0x7fa75c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.450+0000 7fa7611e2700 1 -- 172.21.15.33:0/2233473040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa75c06f7d0 con 0x7fa75c0c4b40 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.450+0000 7fa7611e2700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 0x7fa75c14d9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa74c000c00 tx=0x7fa74c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c01ac00 con 0x7fa75c0c4b40 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4627 (secure 0 0 0) 0x7fa74c009320 con 0x7fa75c0c4b40 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 msgr2=0x7fa75c14d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 0x7fa75c14d9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa74c000c00 tx=0x7fa74c004750 comp rx=0 tx=0).stop 2024-05-08T23:32:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 msgr2=0x7fa75c14d9d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7527fc700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c0c4b40 0x7fa75c14d9d0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa74c000c00 tx=0x7fa74c004750 comp rx=0 tx=0).stop 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 4, 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:32:35.452 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 43, 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.453 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.454 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:32:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.457 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:35.458 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.459 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.461 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:32:35.463 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa74006ac90 msgr2=0x7fa74006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa74006ac90 0x7fa74006d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa754011340 tx=0x7fa75400b040 comp rx=0 tx=0).stop 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 msgr2=0x7fa75c0c57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 --2- 172.21.15.33:0/2233473040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa75c0cb980 0x7fa75c0c57b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa758001eb0 tx=0x7fa75800c7b0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7621e4700 1 -- 172.21.15.33:0/2233473040 reap_dead start 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa7621e4700 1 -- 172.21.15.33:0/2233473040 reap_dead start 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 shutdown_connections 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 >> 172.21.15.33:0/2233473040 conn(0x7fa75c06eb10 msgr2=0x7fa75c0c1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 shutdown_connections 2024-05-08T23:32:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.451+0000 7fa763c47700 1 -- 172.21.15.33:0/2233473040 wait complete. 2024-05-08T23:32:35.465 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:32:35.466 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:32:35.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:32:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.602+0000 7fd6d3c42700 1 Processor -- start 2024-05-08T23:32:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 -- start start 2024-05-08T23:32:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd6cc0c93d0 0x7fd6cc0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc1581a0 con 0x7fd6cc0c6090 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc0c6490 con 0x7fd6cc0666c0 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.603+0000 7fd6d3c42700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc0c97b0 con 0x7fd6cc0c93d0 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49744/0 (socket says 172.21.15.33:49744) 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 -- 172.21.15.33:0/961454078 learned_addr learned my addr 172.21.15.33:0/961454078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d19de700 1 --2- 172.21.15.33:0/961454078 >> v2:172.21.15.42:3300/0 conn(0x7fd6cc0c93d0 0x7fd6cc0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 -- 172.21.15.33:0/961454078 >> v2:172.21.15.42:3300/0 conn(0x7fd6cc0c93d0 msgr2=0x7fd6cc0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/961454078 >> v2:172.21.15.42:3300/0 conn(0x7fd6cc0c93d0 0x7fd6cc0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 -- 172.21.15.33:0/961454078 >> v1:172.21.15.42:6789/0 conn(0x7fd6cc0c6090 legacy=0x7fd6cc15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 -- 172.21.15.33:0/961454078 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc158410 con 0x7fd6cc0666c0 2024-05-08T23:32:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/961454078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06f390 tx=0x7fd6bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e62548a59b012d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d09dc700 1 -- 172.21.15.33:0/961454078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00b870 con 0x7fd6cc0666c0 2024-05-08T23:32:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d09dc700 1 -- 172.21.15.33:0/961454078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc00b9d0 con 0x7fd6cc0666c0 2024-05-08T23:32:35.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.604+0000 7fd6d09dc700 1 -- 172.21.15.33:0/961454078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00bd60 con 0x7fd6cc0666c0 2024-05-08T23:32:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 -- 172.21.15.33:0/961454078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 msgr2=0x7fd6cc0c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/961454078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06f390 tx=0x7fd6bc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 -- 172.21.15.33:0/961454078 shutdown_connections 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/961454078 >> v2:172.21.15.42:3300/0 conn(0x7fd6cc0c93d0 0x7fd6cc0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/961454078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc0c5b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 -- 172.21.15.33:0/961454078 >> 172.21.15.33:0/961454078 conn(0x7fd6cc06a990 msgr2=0x7fd6cc06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 -- 172.21.15.33:0/961454078 shutdown_connections 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.605+0000 7fd6d3c42700 1 -- 172.21.15.33:0/961454078 wait complete. 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.606+0000 7fd6d3c42700 1 Processor -- start 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.606+0000 7fd6d3c42700 1 -- start start 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6cc0c6090 0x7fd6cc154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc0c93d0 0x7fd6cc1ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc153f50 con 0x7fd6cc0666c0 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc1eee70 con 0x7fd6cc0c6090 2024-05-08T23:32:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d3c42700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc1eda00 con 0x7fd6cc0c93d0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d19de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d19de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49748/0 (socket says 172.21.15.33:49748) 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.607+0000 7fd6d19de700 1 -- 172.21.15.33:0/3398329978 learned_addr learned my addr 172.21.15.33:0/3398329978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6cc0c6090 0x7fd6cc154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d21df700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc0c93d0 0x7fd6cc1ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc0c93d0 msgr2=0x7fd6cc1ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc0c93d0 0x7fd6cc1ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6cc0c6090 msgr2=0x7fd6cc154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6cc0c6090 0x7fd6cc154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc009550 con 0x7fd6cc0666c0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d21df700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc0c93d0 0x7fd6cc1ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d19de700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc1553c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e810 tx=0x7fd6c800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8013610 con 0x7fd6cc0666c0 2024-05-08T23:32:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8013770 con 0x7fd6cc0666c0 2024-05-08T23:32:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8013b20 con 0x7fd6cc0666c0 2024-05-08T23:32:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1533d0 con 0x7fd6cc0666c0 2024-05-08T23:32:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.608+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc153990 con 0x7fd6cc0666c0 2024-05-08T23:32:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.609+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 0x7fd6cc1edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.609+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc074900 con 0x7fd6cc1ee720 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.609+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 0x7fd6cc1edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6d11dd700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6cc158cb0 con 0x7fd6cc1ee720 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 0x7fd6cc1edbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06f390 tx=0x7fd6bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd6c8013c80 con 0x7fd6cc0666c0 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6c27fc700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6b006ac90 0x7fd6b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6c8088f50 con 0x7fd6cc0666c0 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.610+0000 7fd6d21df700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6b006ac90 0x7fd6b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0036a0 con 0x7fd6cc1ee720 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd6bc003800 con 0x7fd6cc1ee720 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 msgr2=0x7fd6cc1edbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 0x7fd6cc1edbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06f390 tx=0x7fd6bc004750 comp rx=0 tx=0).stop 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 msgr2=0x7fd6cc1edbc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6c27fc700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc1ee720 0x7fd6cc1edbc0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc06f390 tx=0x7fd6bc004750 comp rx=0 tx=0).stop 2024-05-08T23:32:35.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.611+0000 7fd6d21df700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6b006ac90 0x7fd6b006d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc154310 tx=0x7fd6c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.619+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 0x7fd6cc14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.619+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc148650 con 0x7fd6cc148270 2024-05-08T23:32:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.619+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 0x7fd6cc14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.619+0000 7fd6d11dd700 1 -- 172.21.15.33:0/3398329978 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6cc1589d0 con 0x7fd6cc148270 2024-05-08T23:32:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d11dd700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 0x7fd6cc14acf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009130 tx=0x7fd6bc011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc027050 con 0x7fd6cc148270 2024-05-08T23:32:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4940 (secure 0 0 0) 0x7fd6bc02c070 con 0x7fd6cc148270 2024-05-08T23:32:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 msgr2=0x7fd6cc14acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 0x7fd6cc14acf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009130 tx=0x7fd6bc011000 comp rx=0 tx=0).stop 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 msgr2=0x7fd6cc14acf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6c27fc700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd6cc148270 0x7fd6cc14acf0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009130 tx=0x7fd6bc011000 comp rx=0 tx=0).stop 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 4, 2024-05-08T23:32:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 43, 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:32:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:32:35.634 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6b006ac90 msgr2=0x7fd6b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6b006ac90 0x7fd6b006d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc154310 tx=0x7fd6c4009040 comp rx=0 tx=0).stop 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 msgr2=0x7fd6cc1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d3c42700 1 --2- 172.21.15.33:0/3398329978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6cc0666c0 0x7fd6cc1553c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800e810 tx=0x7fd6c800ebd0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.620+0000 7fd6d21df700 1 -- 172.21.15.33:0/3398329978 reap_dead start 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.621+0000 7fd6d21df700 1 -- 172.21.15.33:0/3398329978 reap_dead start 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.621+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 shutdown_connections 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.621+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 >> 172.21.15.33:0/3398329978 conn(0x7fd6cc06a990 msgr2=0x7fd6cc1569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.621+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 shutdown_connections 2024-05-08T23:32:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.621+0000 7fd6d3c42700 1 -- 172.21.15.33:0/3398329978 wait complete. 2024-05-08T23:32:35.637 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:32:35.637 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-05-08T23:32:35.637 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:32:35.637 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-05-08T23:32:35.637 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T23:32:35.637 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T23:32:35.637 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:35.652 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T23:32:35.652 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:32:35.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:32:35.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.790+0000 7f31dd493700 1 Processor -- start 2024-05-08T23:32:35.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.790+0000 7f31dd493700 1 -- start start 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31dd493700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31d81544f0 0x7f31d80ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31dd493700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 0x7f31d806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31dd493700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d815e720 con 0x7f31d80b3140 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31dd493700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d806f2a0 con 0x7f31d81544f0 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31dd493700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d806e3c0 con 0x7f31d806d1d0 2024-05-08T23:32:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d67fc700 1 -- >> v1:172.21.15.42:6789/0 conn(0x7f31d80b3140 legacy=0x7f31d806eb90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31d81544f0 0x7f31d80ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d77fe700 1 -- >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 msgr2=0x7f31d806d600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d77fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 0x7f31d806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d6ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f31d81544f0 0x7f31d80ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49762/0 (socket says 172.21.15.33:49762) 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.791+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2347633250 learned_addr learned my addr 172.21.15.33:0/2347633250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2347633250 >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 msgr2=0x7f31d806d600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d6ffd700 1 --2- 172.21.15.33:0/2347633250 >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 0x7f31d806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2347633250 >> v1:172.21.15.42:6789/0 conn(0x7f31d80b3140 legacy=0x7f31d806eb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2347633250 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d815d960 con 0x7f31d81544f0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d6ffd700 1 --2- 172.21.15.33:0/2347633250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d81544f0 0x7f31d80ca8c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31d806d0f0 tx=0x7f31cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6167c8b1c3b483f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d5ffb700 1 -- 172.21.15.33:0/2347633250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc004230 con 0x7f31d81544f0 2024-05-08T23:32:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d5ffb700 1 -- 172.21.15.33:0/2347633250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc004390 con 0x7f31d81544f0 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.792+0000 7f31d5ffb700 1 -- 172.21.15.33:0/2347633250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0036a0 con 0x7f31d81544f0 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 -- 172.21.15.33:0/2347633250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d81544f0 msgr2=0x7f31d80ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 --2- 172.21.15.33:0/2347633250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d81544f0 0x7f31d80ca8c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31d806d0f0 tx=0x7f31cc009bd0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 -- 172.21.15.33:0/2347633250 shutdown_connections 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 --2- 172.21.15.33:0/2347633250 >> v2:172.21.15.42:3300/0 conn(0x7f31d806d1d0 0x7f31d806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 --2- 172.21.15.33:0/2347633250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d81544f0 0x7f31d80ca8c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 -- 172.21.15.33:0/2347633250 >> 172.21.15.33:0/2347633250 conn(0x7f31d8066ae0 msgr2=0x7f31d8066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 -- 172.21.15.33:0/2347633250 shutdown_connections 2024-05-08T23:32:35.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.793+0000 7f31dd493700 1 -- 172.21.15.33:0/2347633250 wait complete. 2024-05-08T23:32:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.794+0000 7f31dd493700 1 Processor -- start 2024-05-08T23:32:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.794+0000 7f31dd493700 1 -- start start 2024-05-08T23:32:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 0x7f31d80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f31d81544f0 0x7f31d815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81eddf0 con 0x7f31d806d1d0 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81f0dd0 con 0x7f31d80b3140 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31dd493700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80ae920 con 0x7f31d81544f0 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d67fc700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 msgr2=0x7f31d80cc240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49778/0 (socket says 172.21.15.33:49778) 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2089041465 learned_addr learned my addr 172.21.15.33:0/2089041465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d67fc700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 0x7f31d80cc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d77fe700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f31d81544f0 0x7f31d815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f31d81544f0 msgr2=0x7f31d815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f31d81544f0 0x7f31d815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 msgr2=0x7f31d80cc240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 0x7f31d80cc240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.795+0000 7f31d6ffd700 1 -- 172.21.15.33:0/2089041465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc009550 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31d6ffd700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009130 tx=0x7f31cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc011830 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc0119b0 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31dd493700 1 -- 172.21.15.33:0/2089041465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d80aeb70 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc011d60 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.796+0000 7f31dd493700 1 -- 172.21.15.33:0/2089041465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d80af130 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.797+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b00052f0 con 0x7f31d806d1d0 2024-05-08T23:32:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.797+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f31cc01b440 con 0x7f31d806d1d0 2024-05-08T23:32:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.798+0000 7f31bf7fe700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 0x7f31c406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.798+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31cc08aaa0 con 0x7f31d806d1d0 2024-05-08T23:32:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.798+0000 7f31d67fc700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 0x7f31c406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:35.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.798+0000 7f31d67fc700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 0x7f31c406d100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31d806db50 tx=0x7f31c8009270 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:35.802+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f31cc055d10 con 0x7f31d806d1d0 2024-05-08T23:32:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.131+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f31b00055a0 con 0x7f31d806d1d0 2024-05-08T23:32:36.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.132+0000 7f31bf7fe700 1 -- 172.21.15.33:0/2089041465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f31cc01f620 con 0x7f31d806d1d0 2024-05-08T23:32:36.133 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":43,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:32:36.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 msgr2=0x7f31c406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 0x7f31c406d100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31d806db50 tx=0x7f31c8009270 comp rx=0 tx=0).stop 2024-05-08T23:32:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 msgr2=0x7f31d81f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009130 tx=0x7f31cc004750 comp rx=0 tx=0).stop 2024-05-08T23:32:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 shutdown_connections 2024-05-08T23:32:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f31c406ac40 0x7f31c406d100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f31d81544f0 0x7f31d815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31d806d1d0 0x7f31d81f0890 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 --2- 172.21.15.33:0/2089041465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31d80b3140 0x7f31d80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 >> 172.21.15.33:0/2089041465 conn(0x7f31d8066ae0 msgr2=0x7f31d8152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 shutdown_connections 2024-05-08T23:32:36.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:36.134+0000 7f31bd7fa700 1 -- 172.21.15.33:0/2089041465 wait complete. 2024-05-08T23:32:39.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:32:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.296+0000 7f84ce8d3700 1 Processor -- start 2024-05-08T23:32:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.296+0000 7f84ce8d3700 1 -- start start 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84ce8d3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84c80712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84ce8d3700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c815ad90 con 0x7f84c815b220 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84ce8d3700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c8157ff0 con 0x7f84c806bd10 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84ce8d3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c806c800 con 0x7f84c8157b60 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84c7fff700 1 -- >> v1:172.21.15.42:6789/0 conn(0x7f84c806bd10 legacy=0x7f84c806c0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84c77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f84c815b220 0x7f84c8150fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47096/0 (socket says 172.21.15.33:47096) 2024-05-08T23:32:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84c77fe700 1 -- 172.21.15.33:0/1917318374 learned_addr learned my addr 172.21.15.33:0/1917318374 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.297+0000 7f84cce70700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84c80712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825349570 0 0) 0x7f84c815ad90 con 0x7f84c815b220 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b40035b0 con 0x7f84c815b220 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 693360874 0 0) 0x7f84b40035b0 con 0x7f84c815b220 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 msgr2=0x7f84c80712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84c80712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 >> v1:172.21.15.42:6789/0 conn(0x7f84c806bd10 legacy=0x7f84c806c0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80b2fe0 con 0x7f84c815b220 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f84b0001670 con 0x7f84c815b220 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 >> v1:172.21.15.33:6789/0 conn(0x7f84c815b220 legacy=0x7f84c8150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84b4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4003f90 0x7f84b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b4004d90 0x7f84b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c806c800 con 0x7f84b4004d90 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400dcd0 con 0x7f84b4003f90 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400de60 con 0x7f84c8157b60 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c77fe700 1 -- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4003f90 msgr2=0x7f84b4004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c77fe700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4003f90 0x7f84b4004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b4004d90 0x7f84b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84c7fff700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84b4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 -- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 msgr2=0x7f84b4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8157b60 0x7f84b4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 -- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4003f90 msgr2=0x7f84b4004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4003f90 0x7f84b4004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.298+0000 7f84cce70700 1 -- 172.21.15.33:0/1917318374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40035b0 con 0x7f84b4004d90 2024-05-08T23:32:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84cce70700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b4004d90 0x7f84b4005210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e170 tx=0x7f84b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee31ea38655dd47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000ec90 con 0x7f84b4004d90 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c000f040 con 0x7f84b4004d90 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84c6ffd700 1 -- 172.21.15.33:0/1917318374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0014480 con 0x7f84b4004d90 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 -- 172.21.15.33:0/1917318374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b4004d90 msgr2=0x7f84b4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/1917318374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84b4004d90 0x7f84b4005210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f84c000e170 tx=0x7f84b4005880 comp rx=0 tx=0).stop 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84cce70700 1 -- 172.21.15.33:0/1917318374 reap_dead start 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 -- 172.21.15.33:0/1917318374 shutdown_connections 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 -- 172.21.15.33:0/1917318374 >> 172.21.15.33:0/1917318374 conn(0x7f84c806ce60 msgr2=0x7f84c806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 -- 172.21.15.33:0/1917318374 shutdown_connections 2024-05-08T23:32:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.299+0000 7f84ce8d3700 1 -- 172.21.15.33:0/1917318374 wait complete. 2024-05-08T23:32:39.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.301+0000 7f84ce8d3700 1 Processor -- start 2024-05-08T23:32:39.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.301+0000 7f84ce8d3700 1 -- start start 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 0x7f84c81ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8154d20 0x7f84c80c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814c210 con 0x7f84c8158290 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814d7d0 con 0x7f84b4004d90 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814cf70 con 0x7f84c8154d20 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c7fff700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 msgr2=0x7f84c81ec3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49788/0 (socket says 172.21.15.33:49788) 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 0x7f84c81ec3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 -- 172.21.15.33:0/864678676 learned_addr learned my addr 172.21.15.33:0/864678676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c77fe700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8154d20 0x7f84c80c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 -- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8154d20 msgr2=0x7f84c80c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8154d20 0x7f84c80c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 -- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 msgr2=0x7f84c81ec3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 0x7f84c81ec3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 -- 172.21.15.33:0/864678676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c814d210 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84cce70700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84c0016000 tx=0x7f84c0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0007000 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c000ee80 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c814d4f0 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.302+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0003c60 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.303+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81ebeb0 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.303+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8073770 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.304+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f84c0023020 con 0x7f84c8158290 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.304+0000 7f84c4ff9700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 0x7f84a806d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:39.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.305+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84c008a690 con 0x7f84c8158290 2024-05-08T23:32:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.305+0000 7f84c7fff700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 0x7f84a806d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.305+0000 7f84c7fff700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 0x7f84a806d130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f84b80039c0 tx=0x7f84b8009520 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:39.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.310+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f84c0055900 con 0x7f84c8158290 2024-05-08T23:32:39.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.637+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f84c81eba30 con 0x7f84c8158290 2024-05-08T23:32:39.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.638+0000 7f84c4ff9700 1 -- 172.21.15.33:0/864678676 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f84c81eba30 con 0x7f84c8158290 2024-05-08T23:32:39.638 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":47,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:32:39.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 msgr2=0x7f84a806d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 0x7f84a806d130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f84b80039c0 tx=0x7f84b8009520 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 msgr2=0x7f84c8156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84c0016000 tx=0x7f84c0004b40 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 shutdown_connections 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f84a806ac70 0x7f84a806d130 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c8158290 0x7f84c8156970 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84b4004d90 0x7f84c81ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 --2- 172.21.15.33:0/864678676 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84c8154d20 0x7f84c80c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 >> 172.21.15.33:0/864678676 conn(0x7f84c806ce60 msgr2=0x7f84c806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 shutdown_connections 2024-05-08T23:32:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:39.640+0000 7f84ce8d3700 1 -- 172.21.15.33:0/864678676 wait complete. 2024-05-08T23:32:42.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:32:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.801+0000 7f57af93d700 1 Processor -- start 2024-05-08T23:32:42.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.801+0000 7f57af93d700 1 -- start start 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57af93d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57a815ad80 0x7f57a806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57af93d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a806f810 0x7f57a806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57af93d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a815d870 con 0x7f57a8154460 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57af93d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a8071150 con 0x7f57a815ad80 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57af93d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c7e30 con 0x7f57a806f810 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57aced8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57a815ad80 0x7f57a806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:42.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57ad6d9700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f57a8154460 0x7f57a815dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47108/0 (socket says 172.21.15.33:47108) 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.802+0000 7f57ad6d9700 1 -- 172.21.15.33:0/502352413 learned_addr learned my addr 172.21.15.33:0/502352413 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57adeda700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a806f810 0x7f57a806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57aced8700 1 -- 172.21.15.33:0/502352413 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a806f810 msgr2=0x7f57a806aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57aced8700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a806f810 0x7f57a806aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57aced8700 1 -- 172.21.15.33:0/502352413 >> v1:172.21.15.33:6789/0 conn(0x7f57a8154460 legacy=0x7f57a815dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57aced8700 1 -- 172.21.15.33:0/502352413 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a815b220 con 0x7f57a815ad80 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f57aced8700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a815ad80 0x7f57a806a990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5798009840 tx=0x7f5798009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f6a060cf70653 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f579ffff700 1 -- 172.21.15.33:0/502352413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011830 con 0x7f57a815ad80 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f579ffff700 1 -- 172.21.15.33:0/502352413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798011990 con 0x7f57a815ad80 2024-05-08T23:32:42.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.803+0000 7f579ffff700 1 -- 172.21.15.33:0/502352413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011d20 con 0x7f57a815ad80 2024-05-08T23:32:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 -- 172.21.15.33:0/502352413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a815ad80 msgr2=0x7f57a806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a815ad80 0x7f57a806a990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5798009840 tx=0x7f5798009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 -- 172.21.15.33:0/502352413 shutdown_connections 2024-05-08T23:32:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a806f810 0x7f57a806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 --2- 172.21.15.33:0/502352413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a815ad80 0x7f57a806a990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 -- 172.21.15.33:0/502352413 >> 172.21.15.33:0/502352413 conn(0x7f57a806e990 msgr2=0x7f57a806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 -- 172.21.15.33:0/502352413 shutdown_connections 2024-05-08T23:32:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.804+0000 7f57af93d700 1 -- 172.21.15.33:0/502352413 wait complete. 2024-05-08T23:32:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.805+0000 7f57af93d700 1 Processor -- start 2024-05-08T23:32:42.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.805+0000 7f57af93d700 1 -- start start 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 0x7f57a81f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a815ad80 0x7f57a81f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c6ff0 con 0x7f57a806f810 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c5c70 con 0x7f57a8154460 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57af93d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81e5430 con 0x7f57a815ad80 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57ad6d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57aced8700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 msgr2=0x7f57a81f13e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:42.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57ad6d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49806/0 (socket says 172.21.15.33:49806) 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57ad6d9700 1 -- 172.21.15.33:0/2843835670 learned_addr learned my addr 172.21.15.33:0/2843835670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57aced8700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 0x7f57a81f13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.806+0000 7f57adeda700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a815ad80 0x7f57a81f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 -- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a815ad80 msgr2=0x7f57a81f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a815ad80 0x7f57a81f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 -- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 msgr2=0x7f57a81f13e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 0x7f57a81f13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 -- 172.21.15.33:0/2843835670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980094a0 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57ad6d9700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57a400c490 tx=0x7f57a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4005d70 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a40070d0 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a80c5f30 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4007460 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.807+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81e59d0 con 0x7f57a806f810 2024-05-08T23:32:42.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.808+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8075460 con 0x7f57a806f810 2024-05-08T23:32:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.809+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f57a40075c0 con 0x7f57a806f810 2024-05-08T23:32:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.809+0000 7f579dffb700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 0x7f578c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.809+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57a4088840 con 0x7f57a806f810 2024-05-08T23:32:42.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.809+0000 7f57aced8700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 0x7f578c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:42.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.810+0000 7f57aced8700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 0x7f578c06d100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57a815a870 tx=0x7f5798008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:42.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:42.814+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f57a4053a10 con 0x7f57a806f810 2024-05-08T23:32:43.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.130+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f57a806aff0 con 0x7f57a806f810 2024-05-08T23:32:43.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.130+0000 7f579dffb700 1 -- 172.21.15.33:0/2843835670 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f57a4017080 con 0x7f57a806f810 2024-05-08T23:32:43.131 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":50,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:32:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 msgr2=0x7f578c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 0x7f578c06d100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57a815a870 tx=0x7f5798008030 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 msgr2=0x7f57a80c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57a400c490 tx=0x7f57a400c850 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 shutdown_connections 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f578c06ac40 0x7f578c06d100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57a815ad80 0x7f57a81f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57a806f810 0x7f57a80c6ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 --2- 172.21.15.33:0/2843835670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57a8154460 0x7f57a81f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 >> 172.21.15.33:0/2843835670 conn(0x7f57a806e990 msgr2=0x7f57a8153880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 shutdown_connections 2024-05-08T23:32:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:43.133+0000 7f57af93d700 1 -- 172.21.15.33:0/2843835670 wait complete. 2024-05-08T23:32:46.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.294+0000 7fcc362dc700 1 Processor -- start 2024-05-08T23:32:46.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.294+0000 7fcc362dc700 1 -- start start 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc362dc700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc362dc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc362dc700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3015ad90 con 0x7fcc30157b60 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc362dc700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc30071780 con 0x7fcc3006bd10 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc362dc700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30157fe0 con 0x7fcc300712f0 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc2ffff700 1 -- >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 msgr2=0x7fcc3015b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc2ffff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc2f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcc30157b60 0x7fcc3006c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39144/0 (socket says 172.21.15.33:39144) 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 learned_addr learned my addr 172.21.15.33:0/2186609748 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:32:46.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:46.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:46.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.295+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 msgr2=0x7fcc3015b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:46.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:32:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:32:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:46.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.496+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:32:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 msgr2=0x7fcc3015b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:32:46.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:32:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:46.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:46.897+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:32:47.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:47.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:32:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.698+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 msgr2=0x7fcc3015b220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.698+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:32:47.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.699+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:47.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.699+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:32:47.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.699+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:47.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.699+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:47.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:47.699+0000 7fcc34879700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:32:49.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 msgr2=0x7fcc30150fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc300712f0 0x7fcc30150fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 msgr2=0x7fcc3015b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.42:3300/0 conn(0x7fcc3006bd10 0x7fcc3015b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.33:6789/0 conn(0x7fcc30157b60 legacy=0x7fcc3006c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 -- 172.21.15.33:0/2186609748 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3006c800 con 0x7fcc1c005980 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2dffb700 1 -- 172.21.15.33:0/2186609748 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc30071780 con 0x7fcc1c009610 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.295+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.42:6789/0 conn(0x7fcc1c005980 legacy=0x7fcc1c007de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.296+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.296+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.296+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.296+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:49.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.296+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:49.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.496+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.42:6789/0 conn(0x7fcc1c005980 legacy=0x7fcc1c007de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.497+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.497+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:32:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.497+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.497+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:49.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.497+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:32:49.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.897+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.42:6789/0 conn(0x7fcc1c005980 legacy=0x7fcc1c007de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.898+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.898+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:32:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.898+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:32:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.898+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:32:49.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:49.898+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:32:50.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.698+0000 7fcc2ffff700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.42:6789/0 conn(0x7fcc1c005980 legacy=0x7fcc1c007de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:32:50.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.699+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> v2:172.21.15.33:3300/0 conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.699+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 >> v1:172.21.15.42:6789/0 conn(0x7fcc1c005980 legacy=0x7fcc1c007de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.699+0000 7fcc2f7fe700 1 -- 172.21.15.33:0/2186609748 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c00d1d0 con 0x7fcc1c009610 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.699+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc1c00bad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007db0 tx=0x7fcc24007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abc69cd449c73b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.700+0000 7fcc2effd700 1 -- 172.21.15.33:0/2186609748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24005e10 con 0x7fcc1c009610 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.700+0000 7fcc2effd700 1 -- 172.21.15.33:0/2186609748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24016480 con 0x7fcc1c009610 2024-05-08T23:32:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.700+0000 7fcc2effd700 1 -- 172.21.15.33:0/2186609748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc240167f0 con 0x7fcc1c009610 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 -- 172.21.15.33:0/2186609748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 msgr2=0x7fcc1c00bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc1c00bad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007db0 tx=0x7fcc24007330 comp rx=0 tx=0).stop 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc34879700 1 -- 172.21.15.33:0/2186609748 reap_dead start 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 -- 172.21.15.33:0/2186609748 shutdown_connections 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 --2- 172.21.15.33:0/2186609748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc1c00bad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 -- 172.21.15.33:0/2186609748 >> 172.21.15.33:0/2186609748 conn(0x7fcc3006ce60 msgr2=0x7fcc3006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 -- 172.21.15.33:0/2186609748 shutdown_connections 2024-05-08T23:32:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.701+0000 7fcc362dc700 1 -- 172.21.15.33:0/2186609748 wait complete. 2024-05-08T23:32:50.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.702+0000 7fcc362dc700 1 Processor -- start 2024-05-08T23:32:50.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.703+0000 7fcc362dc700 1 -- start start 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 0x7fcc3014a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc3014a900 0x7fcc301f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3014e990 con 0x7fcc1c009610 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30159d70 con 0x7fcc30154c90 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc362dc700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30155f20 con 0x7fcc3014a900 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2f7fe700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 msgr2=0x7fcc3014a3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 0x7fcc3014a3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55100/0 (socket says 172.21.15.33:55100) 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 -- 172.21.15.33:0/1280516991 learned_addr learned my addr 172.21.15.33:0/1280516991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:50.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc34879700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc3014a900 0x7fcc301f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 -- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc3014a900 msgr2=0x7fcc301f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc3014a900 0x7fcc301f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 -- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 msgr2=0x7fcc3014a3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 0x7fcc3014a3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 -- 172.21.15.33:0/1280516991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2400f040 con 0x7fcc1c009610 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.704+0000 7fcc2ffff700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b980 tx=0x7fcc2000bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.705+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000c9e0 con 0x7fcc1c009610 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.705+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2000a040 con 0x7fcc1c009610 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.705+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc301597b0 con 0x7fcc1c009610 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.705+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000cdc0 con 0x7fcc1c009610 2024-05-08T23:32:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.705+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc301558d0 con 0x7fcc1c009610 2024-05-08T23:32:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.706+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc30073770 con 0x7fcc1c009610 2024-05-08T23:32:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.707+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcc20012020 con 0x7fcc1c009610 2024-05-08T23:32:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.707+0000 7fcc2cff9700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 0x7fcc1c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.707+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc20088730 con 0x7fcc1c009610 2024-05-08T23:32:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.707+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 0x7fcc1c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:50.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.708+0000 7fcc2f7fe700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 0x7fcc1c06de50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc3015a5f0 tx=0x7fcc2400a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:50.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:50.712+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc20053910 con 0x7fcc1c009610 2024-05-08T23:32:51.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.051+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcc30155730 con 0x7fcc1c009610 2024-05-08T23:32:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.051+0000 7fcc2cff9700 1 -- 172.21.15.33:0/1280516991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1532 (secure 0 0 0) 0x7fcc30155730 con 0x7fcc1c009610 2024-05-08T23:32:51.051 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 msgr2=0x7fcc1c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 0x7fcc1c06de50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc3015a5f0 tx=0x7fcc2400a000 comp rx=0 tx=0).stop 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 msgr2=0x7fcc301563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000b980 tx=0x7fcc2000bc90 comp rx=0 tx=0).stop 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 shutdown_connections 2024-05-08T23:32:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc1c06b990 0x7fcc1c06de50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc3014a900 0x7fcc301f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc1c009610 0x7fcc301563b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 --2- 172.21.15.33:0/1280516991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30154c90 0x7fcc3014a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 >> 172.21.15.33:0/1280516991 conn(0x7fcc3006ce60 msgr2=0x7fcc30150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 shutdown_connections 2024-05-08T23:32:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.054+0000 7fcc362dc700 1 -- 172.21.15.33:0/1280516991 wait complete. 2024-05-08T23:32:51.067 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:32:51.067 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-08T23:32:51.067 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:32:51.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.205+0000 7fd9ac449700 1 Processor -- start 2024-05-08T23:32:51.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.206+0000 7fd9ac449700 1 -- start start 2024-05-08T23:32:51.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9ac449700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 0x7fd9a415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9ac449700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 0x7fd9a40d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9ac449700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a40c5c20 con 0x7fd9a40c4c70 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9ac449700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a40b30b0 con 0x7fd9a40d00b0 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9ac449700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4154b50 con 0x7fd9a4154770 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa9e6700 1 -- >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 msgr2=0x7fd9a40d0540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9a99e4700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd9a40c4c70 0x7fd9a40c60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39146/0 (socket says 172.21.15.33:39146) 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa9e6700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 0x7fd9a40d0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9a99e4700 1 -- 172.21.15.33:0/3963095981 learned_addr learned my addr 172.21.15.33:0/3963095981 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/3963095981 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 0x7fd9a415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9a91e3700 1 -- 172.21.15.33:0/3963095981 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045208013 0 0) 0x7fd9a40c5c20 con 0x7fd9a40c4c70 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9a91e3700 1 -- 172.21.15.33:0/3963095981 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9900035b0 con 0x7fd9a40c4c70 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 -- 172.21.15.33:0/3963095981 >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 msgr2=0x7fd9a40d0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/3963095981 >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 0x7fd9a40d0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 -- 172.21.15.33:0/3963095981 >> v1:172.21.15.33:6789/0 conn(0x7fd9a40c4c70 legacy=0x7fd9a40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 -- 172.21.15.33:0/3963095981 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40caed0 con 0x7fd9a4154770 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.207+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/3963095981 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 0x7fd9a415ae50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fafe39f0a526e642 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9a91e3700 1 -- 172.21.15.33:0/3963095981 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998011830 con 0x7fd9a4154770 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9a91e3700 1 -- 172.21.15.33:0/3963095981 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd998011990 con 0x7fd9a4154770 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9a91e3700 1 -- 172.21.15.33:0/3963095981 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998011d20 con 0x7fd9a4154770 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 -- 172.21.15.33:0/3963095981 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 msgr2=0x7fd9a415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 --2- 172.21.15.33:0/3963095981 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 0x7fd9a415ae50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 -- 172.21.15.33:0/3963095981 shutdown_connections 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 --2- 172.21.15.33:0/3963095981 >> v2:172.21.15.42:3300/0 conn(0x7fd9a40d00b0 0x7fd9a40d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 --2- 172.21.15.33:0/3963095981 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a4154770 0x7fd9a415ae50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 -- 172.21.15.33:0/3963095981 >> 172.21.15.33:0/3963095981 conn(0x7fd9a4066ae0 msgr2=0x7fd9a4066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 -- 172.21.15.33:0/3963095981 shutdown_connections 2024-05-08T23:32:51.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.208+0000 7fd9ac449700 1 -- 172.21.15.33:0/3963095981 wait complete. 2024-05-08T23:32:51.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 Processor -- start 2024-05-08T23:32:51.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 -- start start 2024-05-08T23:32:51.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9a40c4c70 0x7fd9a41f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a40d00b0 0x7fd9a40ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 0x7fd9a41590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4156fa0 con 0x7fd9a4154770 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4154d40 con 0x7fd9a40c4c70 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.210+0000 7fd9ac449700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4157520 con 0x7fd9a40d00b0 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa1e5700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9a40c4c70 msgr2=0x7fd9a41f2ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 0x7fd9a41590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa1e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9a40c4c70 0x7fd9a41f2ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 0x7fd9a41590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55108/0 (socket says 172.21.15.33:55108) 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 learned_addr learned my addr 172.21.15.33:0/696018837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9a99e4700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a40d00b0 0x7fd9a40ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a40d00b0 msgr2=0x7fd9a40ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9a40d00b0 0x7fd9a40ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9a40c4c70 msgr2=0x7fd9a41f2ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9a40c4c70 0x7fd9a41f2ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9980094a0 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9aa9e6700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 0x7fd9a41590b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00b990 tx=0x7fd99c00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00d8e0 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c00da40 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a4154f90 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00de10 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.211+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a41571b0 con 0x7fd9a4154770 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.212+0000 7fd9ac449700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 0x7fd9a4147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.212+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4074690 con 0x7fd9a41578b0 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.212+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 0x7fd9a4147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd9aa1e5700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9a41554b0 con 0x7fd9a41578b0 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 0x7fd9a4147350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd99c01e3f0 con 0x7fd9a4154770 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd996ffd700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd98806ac40 0x7fd98806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980036a0 con 0x7fd9a41578b0 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd9a99e4700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd98806ac40 0x7fd98806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.213+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd99c0893e0 con 0x7fd9a4154770 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd99801e070 con 0x7fd9a41578b0 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 msgr2=0x7fd9a4147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd996ffd700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 0x7fd9a4147350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd998009840 tx=0x7fd998004750 comp rx=0 tx=0).stop 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 msgr2=0x7fd9a4147350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd996ffd700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a41578b0 0x7fd9a4147350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.214+0000 7fd9a99e4700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd98806ac40 0x7fd98806d100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ca0e0 tx=0x7fd9a0007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd9ac449700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 0x7fd9a41494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4157c90 con 0x7fd9a4147890 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 0x7fd9a41494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd9aa1e5700 1 -- 172.21.15.33:0/696018837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd9a40c52f0 con 0x7fd9a4147890 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd9aa1e5700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 0x7fd9a41494b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.222+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99802b680 con 0x7fd9a4147890 2024-05-08T23:32:51.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4957 (secure 0 0 0) 0x7fd998024400 con 0x7fd9a4147890 2024-05-08T23:32:51.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 msgr2=0x7fd9a41494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd996ffd700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 0x7fd9a41494b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998009210 comp rx=0 tx=0).stop 2024-05-08T23:32:51.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd996ffd700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 msgr2=0x7fd9a41494b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd996ffd700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4147890 0x7fd9a41494b0 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd998009fd0 tx=0x7fd998009210 comp rx=0 tx=0).stop 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 8, 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.223 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:32:51.226 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:51.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.233 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.234 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:32:51.235 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd98806ac40 msgr2=0x7fd98806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd98806ac40 0x7fd98806d100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40ca0e0 tx=0x7fd9a0007dc0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 msgr2=0x7fd9a41590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 --2- 172.21.15.33:0/696018837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9a4154770 0x7fd9a41590b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00b990 tx=0x7fd99c00bca0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 reap_dead start 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9aa9e6700 1 -- 172.21.15.33:0/696018837 reap_dead start 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 shutdown_connections 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 >> 172.21.15.33:0/696018837 conn(0x7fd9a4066ae0 msgr2=0x7fd9a4152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 shutdown_connections 2024-05-08T23:32:51.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.223+0000 7fd9ac449700 1 -- 172.21.15.33:0/696018837 wait complete. 2024-05-08T23:32:51.238 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:32:51.238 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:32:51.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:32:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.376+0000 7fa02b7ce700 1 Processor -- start 2024-05-08T23:32:51.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.377+0000 7fa02b7ce700 1 -- start start 2024-05-08T23:32:51.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.377+0000 7fa02b7ce700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0240c8bc0 0x7fa024157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.377+0000 7fa02b7ce700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa024150e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.377+0000 7fa02b7ce700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240c94c0 con 0x7fa0240c8fa0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02b7ce700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa024157ff0 con 0x7fa0240c8bc0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02b7ce700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024071480 con 0x7fa0240668c0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa0240c8bc0 0x7fa024157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa028d69700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa0240c8fa0 0x7fa0240c49c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39154/0 (socket says 172.21.15.33:39154) 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa028d69700 1 -- 172.21.15.33:0/114065876 learned_addr learned my addr 172.21.15.33:0/114065876 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa029d6b700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa024150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 -- 172.21.15.33:0/114065876 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 msgr2=0x7fa024150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa024150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 -- 172.21.15.33:0/114065876 >> v1:172.21.15.33:6789/0 conn(0x7fa0240c8fa0 legacy=0x7fa0240c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 -- 172.21.15.33:0/114065876 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024071760 con 0x7fa0240c8bc0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa01bfff700 1 -- 172.21.15.33:0/114065876 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206964626 0 0) 0x7fa0240c94c0 con 0x7fa0240c8fa0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa02956a700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa024157ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa020009840 tx=0x7fa020009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b5a89b6fcd08ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa01bfff700 1 -- 172.21.15.33:0/114065876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020011830 con 0x7fa0240c8bc0 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa01bfff700 1 -- 172.21.15.33:0/114065876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020011990 con 0x7fa0240c8bc0 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.378+0000 7fa01bfff700 1 -- 172.21.15.33:0/114065876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020011d20 con 0x7fa0240c8bc0 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 -- 172.21.15.33:0/114065876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 msgr2=0x7fa024157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa024157ab0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa020009840 tx=0x7fa020009b50 comp rx=0 tx=0).stop 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 -- 172.21.15.33:0/114065876 shutdown_connections 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa024150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/114065876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa024157ab0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 -- 172.21.15.33:0/114065876 >> 172.21.15.33:0/114065876 conn(0x7fa024067370 msgr2=0x7fa02406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 -- 172.21.15.33:0/114065876 shutdown_connections 2024-05-08T23:32:51.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.379+0000 7fa02b7ce700 1 -- 172.21.15.33:0/114065876 wait complete. 2024-05-08T23:32:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.380+0000 7fa02b7ce700 1 Processor -- start 2024-05-08T23:32:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 -- start start 2024-05-08T23:32:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa0241e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa02406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0240c8fa0 0x7fa0241ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241f3ab0 con 0x7fa0240c8bc0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241f2c00 con 0x7fa0240c8fa0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa02b7ce700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241f2ea0 con 0x7fa0240668c0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa028d69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa02406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa029d6b700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0240c8fa0 msgr2=0x7fa0241ef9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa029d6b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0240c8fa0 0x7fa0241ef9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa028d69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa02406e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55150/0 (socket says 172.21.15.33:55150) 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.381+0000 7fa028d69700 1 -- 172.21.15.33:0/2595045223 learned_addr learned my addr 172.21.15.33:0/2595045223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa02956a700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa0241e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 msgr2=0x7fa0241e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa0240668c0 0x7fa0241e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0240c8fa0 msgr2=0x7fa0241ef9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0240c8fa0 0x7fa0241ef9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200094a0 con 0x7fa0240c8bc0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa028d69700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa02406e2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c520 tx=0x7fa01400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014004c80 con 0x7fa0240c8bc0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014004de0 con 0x7fa0240c8bc0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.382+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140072d0 con 0x7fa0240c8bc0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.383+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0241f3180 con 0x7fa0240c8bc0 2024-05-08T23:32:51.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.383+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0241f3740 con 0x7fa0240c8bc0 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.384+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 0x7fa0241efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.384+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024074930 con 0x7fa024149c80 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.384+0000 7fa02956a700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 0x7fa0241efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.384+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa014007430 con 0x7fa0240c8bc0 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.384+0000 7fa02956a700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa024072000 con 0x7fa024149c80 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa02956a700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 0x7fa0241efee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa020009840 tx=0x7fa020004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa00806acc0 0x7fa00806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa014088f80 con 0x7fa0240c8bc0 2024-05-08T23:32:51.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa029d6b700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa00806acc0 0x7fa00806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02001f070 con 0x7fa024149c80 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa020004230 con 0x7fa024149c80 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 msgr2=0x7fa0241efee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 0x7fa0241efee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa020009840 tx=0x7fa020004750 comp rx=0 tx=0).stop 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 msgr2=0x7fa0241efee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa019ffb700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149c80 0x7fa0241efee0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa020009840 tx=0x7fa020004750 comp rx=0 tx=0).stop 2024-05-08T23:32:51.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.385+0000 7fa029d6b700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa00806acc0 0x7fa00806d180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009bb0 tx=0x7fa01c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.393+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 0x7fa024149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.393+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024071d20 con 0x7fa024149040 2024-05-08T23:32:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.393+0000 7fa02956a700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 0x7fa024149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa02956a700 1 -- 172.21.15.33:0/2595045223 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0240722e0 con 0x7fa024149040 2024-05-08T23:32:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa02956a700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 0x7fa024149420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa020009fd0 tx=0x7fa020011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020003ec0 con 0x7fa024149040 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4928 (secure 0 0 0) 0x7fa020031070 con 0x7fa024149040 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 msgr2=0x7fa024149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 0x7fa024149420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa020009fd0 tx=0x7fa020011000 comp rx=0 tx=0).stop 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 msgr2=0x7fa024149420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.394+0000 7fa019ffb700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa024149040 0x7fa024149420 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa020009fd0 tx=0x7fa020011000 comp rx=0 tx=0).stop 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:51.394 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 8, 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.396 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:32:51.397 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:32:51.398 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:32:51.399 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:32:51.400 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:32:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:32:51.404 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:32:51.406 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:32:51.407 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa00806acc0 msgr2=0x7fa00806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa00806acc0 0x7fa00806d180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009bb0 tx=0x7fa01c00b040 comp rx=0 tx=0).stop 2024-05-08T23:32:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 msgr2=0x7fa02406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 --2- 172.21.15.33:0/2595045223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0240c8bc0 0x7fa02406e2c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa01400c520 tx=0x7fa01400c8e0 comp rx=0 tx=0).stop 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa029d6b700 1 -- 172.21.15.33:0/2595045223 reap_dead start 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa029d6b700 1 -- 172.21.15.33:0/2595045223 reap_dead start 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 shutdown_connections 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 >> 172.21.15.33:0/2595045223 conn(0x7fa024067370 msgr2=0x7fa024150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 shutdown_connections 2024-05-08T23:32:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:32:51.395+0000 7fa02b7ce700 1 -- 172.21.15.33:0/2595045223 wait complete. 2024-05-08T23:32:51.409 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:32:51.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:33:11.410 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T23:33:11.410 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-05-08T23:33:11.410 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T23:33:11.411 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T23:33:11.414 INFO:tasks.ceph.mon.b:Started 2024-05-08T23:33:11.414 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:33:11.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.560+0000 7ff297bbb700 1 Processor -- start 2024-05-08T23:33:11.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.560+0000 7ff297bbb700 1 -- start start 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff297bbb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff297bbb700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff297bbb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2900cdd30 con 0x7ff2900c8a50 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff297bbb700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff290070410 con 0x7ff2900c38a0 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff297bbb700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff290158610 con 0x7ff2900c8e30 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57562/0 (socket says 172.21.15.33:57562) 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 learned_addr learned my addr 172.21.15.33:0/2579283278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295957700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2900c8a50 0x7ff290151e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40320/0 (socket says 172.21.15.33:40320) 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:11.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:11.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.561+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:11.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:11.762+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.163+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff295156700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:12.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:12.964+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 msgr2=0x7ff290162960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.42:3300/0 conn(0x7ff2900c8e30 0x7ff290162960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 -- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 msgr2=0x7ff290158aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 --2- 172.21.15.33:0/2579283278 >> v2:172.21.15.33:3300/0 conn(0x7ff2900c38a0 0x7ff290158aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 -- 172.21.15.33:0/2579283278 >> v1:172.21.15.33:6789/0 conn(0x7ff2900c8a50 legacy=0x7ff290151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 -- 172.21.15.33:0/2579283278 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff290158fe0 con 0x7ff274006270 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.561+0000 7ff2877fe700 1 -- 172.21.15.33:0/2579283278 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290070410 con 0x7ff274003740 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.562+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.562+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:14.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.562+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.562+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.562+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:14.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.763+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:14.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.763+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:14.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.763+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:14.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.763+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:14.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:14.763+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.164+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.164+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.164+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.164+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.164+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.965+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.965+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.965+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.965+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:15.965+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:33:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725008135 0 0) 0x7ff290158fe0 con 0x7ff274006270 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff27c0035b0 con 0x7ff274006270 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010206117 0 0) 0x7ff27c0035b0 con 0x7ff274006270 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff274005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff274005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27400b0c0 con 0x7ff274006270 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7ff28000c570 con 0x7ff274006270 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 >> v1:172.21.15.42:6789/0 conn(0x7ff274006270 legacy=0x7ff274008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff296158700 1 -- 172.21.15.33:0/2579283278 reap_dead start 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.598+0000 7ff294955700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff27c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff294955700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 0x7ff27c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff294955700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff27c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290070410 con 0x7ff27c004060 2024-05-08T23:33:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c00bcf0 con 0x7ff27c004de0 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0102c0 con 0x7ff274003740 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 0x7ff27c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff27c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff27c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 msgr2=0x7ff27c005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff27c005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 msgr2=0x7ff27c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 0x7ff27c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff295957700 1 -- 172.21.15.33:0/2579283278 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0035b0 con 0x7ff274003740 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.599+0000 7ff296158700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff27c005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.600+0000 7ff295156700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 0x7ff27c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.600+0000 7ff295957700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff27c003a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff28c007cf0 tx=0x7ff27c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7e1dc679b2e876 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.601+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017460 con 0x7ff274003740 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.601+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0175e0 con 0x7ff274003740 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.601+0000 7ff294955700 1 -- 172.21.15.33:0/2579283278 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017990 con 0x7ff274003740 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 -- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 msgr2=0x7ff27c003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff27c003a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff28c007cf0 tx=0x7ff27c005880 comp rx=0 tx=0).stop 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 -- 172.21.15.33:0/2579283278 shutdown_connections 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff27c005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff27c004060 0x7ff27c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 --2- 172.21.15.33:0/2579283278 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff274003740 0x7ff27c003a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 -- 172.21.15.33:0/2579283278 >> 172.21.15.33:0/2579283278 conn(0x7ff290072230 msgr2=0x7ff290072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 -- 172.21.15.33:0/2579283278 shutdown_connections 2024-05-08T23:33:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.602+0000 7ff297bbb700 1 -- 172.21.15.33:0/2579283278 wait complete. 2024-05-08T23:33:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.605+0000 7ff297bbb700 1 Processor -- start 2024-05-08T23:33:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.605+0000 7ff297bbb700 1 -- start start 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.605+0000 7ff297bbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.605+0000 7ff297bbb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff27c004060 0x7ff290155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff297bbb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff2900cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff297bbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290151110 con 0x7ff274003740 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff297bbb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901500e0 con 0x7ff27c004de0 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff297bbb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901561b0 con 0x7ff27c004060 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51474/0 (socket says 172.21.15.33:51474) 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 -- 172.21.15.33:0/3852786254 learned_addr learned my addr 172.21.15.33:0/3852786254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295156700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff27c004060 0x7ff290155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff296158700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff2900cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 -- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff27c004060 msgr2=0x7ff290155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff27c004060 0x7ff290155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 -- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 msgr2=0x7ff2900cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff2900cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 -- 172.21.15.33:0/3852786254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290150b70 con 0x7ff274003740 2024-05-08T23:33:16.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.606+0000 7ff295957700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff27c010450 tx=0x7ff28c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.607+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017460 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.607+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c000ed0 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.607+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff290150e30 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.607+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c020880 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.607+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2901eecb0 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.608+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2901ef300 con 0x7ff274003740 2024-05-08T23:33:16.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.608+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff28c0209e0 con 0x7ff274003740 2024-05-08T23:33:16.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.609+0000 7ff2867fc700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 0x7ff27406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:16.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.609+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff28c08ac50 con 0x7ff274003740 2024-05-08T23:33:16.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.609+0000 7ff295156700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 0x7ff27406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.610+0000 7ff295156700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 0x7ff27406d960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff280006e50 tx=0x7ff280008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.614+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff28c090020 con 0x7ff274003740 2024-05-08T23:33:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.953+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff2901ee830 con 0x7ff274003740 2024-05-08T23:33:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.953+0000 7ff2867fc700 1 -- 172.21.15.33:0/3852786254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7ff2901ee830 con 0x7ff274003740 2024-05-08T23:33:16.952 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 msgr2=0x7ff27406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 0x7ff27406d960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff280006e50 tx=0x7ff280008040 comp rx=0 tx=0).stop 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 msgr2=0x7ff29014ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff27c010450 tx=0x7ff28c001120 comp rx=0 tx=0).stop 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 shutdown_connections 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff27406b4a0 0x7ff27406d960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff27c004de0 0x7ff2900cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff274003740 0x7ff29014ee40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 --2- 172.21.15.33:0/3852786254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff27c004060 0x7ff290155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 >> 172.21.15.33:0/3852786254 conn(0x7ff290072230 msgr2=0x7ff2901598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 shutdown_connections 2024-05-08T23:33:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:16.956+0000 7ff297bbb700 1 -- 172.21.15.33:0/3852786254 wait complete. 2024-05-08T23:33:16.969 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:33:16.969 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:33:16.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:33:17.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.110+0000 7ff125ae9700 1 Processor -- start 2024-05-08T23:33:17.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.110+0000 7ff125ae9700 1 -- start start 2024-05-08T23:33:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff125ae9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1200c38a0 0x7ff1200c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff125ae9700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff120070450 con 0x7ff1200c9a60 2024-05-08T23:33:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff125ae9700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff120070730 con 0x7ff120152350 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff125ae9700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1200c2e90 con 0x7ff1200c38a0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1200c38a0 0x7ff1200c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11ffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1200c9a60 0x7ff1200c85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54462/0 (socket says 172.21.15.33:54462) 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11ffff700 1 -- 172.21.15.33:0/1181914721 learned_addr learned my addr 172.21.15.33:0/1181914721 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11effd700 1 -- 172.21.15.33:0/1181914721 >> v1:172.21.15.42:6789/0 conn(0x7ff120152350 legacy=0x7ff120158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11effd700 1 -- 172.21.15.33:0/1181914721 >> v1:172.21.15.33:6789/0 conn(0x7ff1200c9a60 legacy=0x7ff1200c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11effd700 1 -- 172.21.15.33:0/1181914721 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120070a10 con 0x7ff1200c38a0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.111+0000 7ff11e7fc700 1 -- 172.21.15.33:0/1181914721 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140702190 0 0) 0x7ff120070450 con 0x7ff1200c9a60 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff11effd700 1 --2- 172.21.15.33:0/1181914721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c38a0 0x7ff1200c9520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96640352269e9328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff11e7fc700 1 -- 172.21.15.33:0/1181914721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011830 con 0x7ff1200c38a0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff11e7fc700 1 -- 172.21.15.33:0/1181914721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108011990 con 0x7ff1200c38a0 2024-05-08T23:33:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff11e7fc700 1 -- 172.21.15.33:0/1181914721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108011d20 con 0x7ff1200c38a0 2024-05-08T23:33:17.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 -- 172.21.15.33:0/1181914721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c38a0 msgr2=0x7ff1200c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 --2- 172.21.15.33:0/1181914721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c38a0 0x7ff1200c9520 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff108009840 tx=0x7ff108009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 -- 172.21.15.33:0/1181914721 shutdown_connections 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 --2- 172.21.15.33:0/1181914721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c38a0 0x7ff1200c9520 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 -- 172.21.15.33:0/1181914721 >> 172.21.15.33:0/1181914721 conn(0x7ff1200666c0 msgr2=0x7ff120066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 -- 172.21.15.33:0/1181914721 shutdown_connections 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.112+0000 7ff125ae9700 1 -- 172.21.15.33:0/1181914721 wait complete. 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.114+0000 7ff125ae9700 1 Processor -- start 2024-05-08T23:33:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.114+0000 7ff125ae9700 1 -- start start 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1200c38a0 0x7ff120152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 0x7ff120155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff120152350 0x7ff120155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201f04e0 con 0x7ff1200c9a60 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201f0a10 con 0x7ff1200c38a0 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff125ae9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201546d0 con 0x7ff120152350 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 0x7ff120155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 0x7ff120155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51494/0 (socket says 172.21.15.33:51494) 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 -- 172.21.15.33:0/2698526366 learned_addr learned my addr 172.21.15.33:0/2698526366 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1200c38a0 0x7ff120152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11ffff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff120152350 0x7ff120155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff120152350 msgr2=0x7ff120155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff120152350 0x7ff120155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1200c38a0 msgr2=0x7ff120152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1200c38a0 0x7ff120152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080094a0 con 0x7ff1200c9a60 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1200c38a0 0x7ff120152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.115+0000 7ff11effd700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 0x7ff120155050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1080090b0 tx=0x7ff108004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.116+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10801a900 con 0x7ff1200c9a60 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.116+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10801aa60 con 0x7ff1200c9a60 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.116+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120155f10 con 0x7ff1200c9a60 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.116+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10801add0 con 0x7ff1200c9a60 2024-05-08T23:33:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.116+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff120154360 con 0x7ff1200c9a60 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff125ae9700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 0x7ff1201f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12006b420 con 0x7ff1200c7df0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 0x7ff1201f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff11f7fe700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff120070110 con 0x7ff1200c7df0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 0x7ff1201f2f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efd0 tx=0x7ff11000c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.117+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff108022070 con 0x7ff1200c9a60 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff10006ac30 0x7ff10006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff10808a590 con 0x7ff1200c9a60 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff11ffff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff10006ac30 0x7ff10006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110002100 con 0x7ff1200c7df0 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff110003c10 con 0x7ff1200c7df0 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 msgr2=0x7ff1201f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 0x7ff1201f2f80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efd0 tx=0x7ff11000c3c0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 msgr2=0x7ff1201f2f80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff117fff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c7df0 0x7ff1201f2f80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.118+0000 7ff11ffff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff10006ac30 0x7ff10006d0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff118000f60 tx=0x7ff118007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.126+0000 7ff125ae9700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 0x7ff1201485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.126+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1200c3380 con 0x7ff120153e20 2024-05-08T23:33:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.126+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 0x7ff1201485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff11f7fe700 1 -- 172.21.15.33:0/2698526366 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1200b3f10 con 0x7ff120153e20 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff11f7fe700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 0x7ff1201485b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efd0 tx=0x7ff110001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1100052f0 con 0x7ff120153e20 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4969 (secure 0 0 0) 0x7ff110005450 con 0x7ff120153e20 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 msgr2=0x7ff1201485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 0x7ff1201485b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efd0 tx=0x7ff110001400 comp rx=0 tx=0).stop 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 msgr2=0x7ff1201485b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.127+0000 7ff117fff700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120153e20 0x7ff1201485b0 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efd0 tx=0x7ff110001400 comp rx=0 tx=0).stop 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:33:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.128 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:17.129 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:17.130 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:17.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:17.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:17.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.139 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:17.140 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff10006ac30 msgr2=0x7ff10006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff10006ac30 0x7ff10006d0f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff118000f60 tx=0x7ff118007b60 comp rx=0 tx=0).stop 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 msgr2=0x7ff120155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 --2- 172.21.15.33:0/2698526366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1200c9a60 0x7ff120155050 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1080090b0 tx=0x7ff108004750 comp rx=0 tx=0).stop 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff11ffff700 1 -- 172.21.15.33:0/2698526366 reap_dead start 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff11ffff700 1 -- 172.21.15.33:0/2698526366 reap_dead start 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 shutdown_connections 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 >> 172.21.15.33:0/2698526366 conn(0x7ff1200666c0 msgr2=0x7ff12014acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 shutdown_connections 2024-05-08T23:33:17.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.128+0000 7ff125ae9700 1 -- 172.21.15.33:0/2698526366 wait complete. 2024-05-08T23:33:17.142 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:17.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:33:17.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:33:17.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:17.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:17.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.295+0000 7f5cdc10b700 1 Processor -- start 2024-05-08T23:33:17.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.296+0000 7f5cdc10b700 1 -- start start 2024-05-08T23:33:17.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.296+0000 7f5cdc10b700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f5cd4071700 0x7f5cd415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.296+0000 7f5cdc10b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5cd415bb30 0x7f5cd4151d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cdc10b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40c79f0 con 0x7f5cd4071320 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cdc10b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd4158a90 con 0x7f5cd415bb30 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cdc10b700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd406faa0 con 0x7f5cd4071700 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5cd415bb30 0x7f5cd4151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cd9ea7700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5cd4071320 0x7f5cd406b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54474/0 (socket says 172.21.15.33:54474) 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cd9ea7700 1 -- 172.21.15.33:0/1642844525 learned_addr learned my addr 172.21.15.33:0/1642844525 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/1642844525 >> v2:172.21.15.42:3300/0 conn(0x7f5cd4071700 0x7f5cd415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 -- 172.21.15.33:0/1642844525 >> v2:172.21.15.42:3300/0 conn(0x7f5cd4071700 msgr2=0x7f5cd415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/1642844525 >> v2:172.21.15.42:3300/0 conn(0x7f5cd4071700 0x7f5cd415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 -- 172.21.15.33:0/1642844525 >> v1:172.21.15.33:6789/0 conn(0x7f5cd4071320 legacy=0x7f5cd406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 -- 172.21.15.33:0/1642844525 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd41581a0 con 0x7f5cd415bb30 2024-05-08T23:33:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cd8ea5700 1 -- 172.21.15.33:0/1642844525 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827611665 0 0) 0x7f5cd40c79f0 con 0x7f5cd4071320 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/1642844525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd415bb30 0x7f5cd4151d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cd406e320 tx=0x7f5ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602dfe32d660fcff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.297+0000 7f5cd8ea5700 1 -- 172.21.15.33:0/1642844525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00e070 con 0x7f5cd415bb30 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cd8ea5700 1 -- 172.21.15.33:0/1642844525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc00bbd0 con 0x7f5cd415bb30 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cd8ea5700 1 -- 172.21.15.33:0/1642844525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc012430 con 0x7f5cd415bb30 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 -- 172.21.15.33:0/1642844525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd415bb30 msgr2=0x7f5cd4151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/1642844525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd415bb30 0x7f5cd4151d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cd406e320 tx=0x7f5ccc00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 -- 172.21.15.33:0/1642844525 shutdown_connections 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/1642844525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd415bb30 0x7f5cd4151d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/1642844525 >> v2:172.21.15.42:3300/0 conn(0x7f5cd4071700 0x7f5cd415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 -- 172.21.15.33:0/1642844525 >> 172.21.15.33:0/1642844525 conn(0x7f5cd406e990 msgr2=0x7f5cd406ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 -- 172.21.15.33:0/1642844525 shutdown_connections 2024-05-08T23:33:17.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.298+0000 7f5cdc10b700 1 -- 172.21.15.33:0/1642844525 wait complete. 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.300+0000 7f5cdc10b700 1 Processor -- start 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.300+0000 7f5cdc10b700 1 -- start start 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 0x7f5cd41e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4071700 0x7f5cd414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5cd415bb30 0x7f5cd41e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41e1410 con 0x7f5cd4071320 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41e16f0 con 0x7f5cd4071700 2024-05-08T23:33:17.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cdc10b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd41e20c0 con 0x7f5cd415bb30 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cd9ea7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 0x7f5cd41e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cd9ea7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 0x7f5cd41e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51524/0 (socket says 172.21.15.33:51524) 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.301+0000 7f5cd9ea7700 1 -- 172.21.15.33:0/633274646 learned_addr learned my addr 172.21.15.33:0/633274646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4071700 0x7f5cd414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5cd415bb30 0x7f5cd41e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5cd415bb30 msgr2=0x7f5cd41e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5cd415bb30 0x7f5cd41e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4071700 msgr2=0x7f5cd414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4071700 0x7f5cd414ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc009a70 con 0x7f5cd4071320 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4071700 0x7f5cd414ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:17.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5cd415bb30 0x7f5cd41e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cd9ea7700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 0x7f5cd41e1b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c750 tx=0x7f5cd000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004c80 con 0x7f5cd4071320 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0004e00 con 0x7f5cd4071320 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40c5f50 con 0x7f5cd4071320 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0007310 con 0x7f5cd4071320 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.302+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40c6510 con 0x7f5cd4071320 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.303+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 0x7f5cd41f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.303+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40749b0 con 0x7f5cd41e41f0 2024-05-08T23:33:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.303+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 0x7f5cd41f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cd96a6700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cd4158440 con 0x7f5cd41e41f0 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5cd0007470 con 0x7f5cd4071320 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 0x7f5cd41f16a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40c5d60 tx=0x7f5cc800f8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cc67fc700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5cb806acc0 0x7f5cb806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f5cd0089140 con 0x7f5cd4071320 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.304+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5cb806acc0 0x7f5cb806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004020 con 0x7f5cd41e41f0 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5cc800c030 con 0x7f5cd41e41f0 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 msgr2=0x7f5cd41f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 0x7f5cd41f16a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40c5d60 tx=0x7f5cc800f8e0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 msgr2=0x7f5cd41f16a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cc67fc700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41e41f0 0x7f5cd41f16a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.305+0000 7f5cda6a8700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5cb806acc0 0x7f5cb806d180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cd415bf30 tx=0x7f5ccc011000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.315+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 0x7f5cd41593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.315+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd406f7c0 con 0x7f5cd41ed960 2024-05-08T23:33:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.315+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 0x7f5cd41593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.316+0000 7f5cd96a6700 1 -- 172.21.15.33:0/633274646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5cd4158720 con 0x7f5cd41ed960 2024-05-08T23:33:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.316+0000 7f5cd96a6700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 0x7f5cd41593d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cd414dd60 tx=0x7f5cc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.316+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8003ca0 con 0x7f5cd41ed960 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4457 (secure 0 0 0) 0x7f5cc8003e00 con 0x7f5cd41ed960 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 msgr2=0x7f5cd41593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cc67fc700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 0x7f5cd41593d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cd414dd60 tx=0x7f5cc8004750 comp rx=0 tx=0).stop 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cc67fc700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 msgr2=0x7f5cd41593d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cc67fc700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd41ed960 0x7f5cd41593d0 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5cd414dd60 tx=0x7f5cc8004750 comp rx=0 tx=0).stop 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:33:17.316 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.317 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.318 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.320 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.321 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.322 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.324 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:17.327 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5cb806acc0 msgr2=0x7f5cb806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5cb806acc0 0x7f5cb806d180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5cd415bf30 tx=0x7f5ccc011000 comp rx=0 tx=0).stop 2024-05-08T23:33:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 msgr2=0x7f5cd41e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 --2- 172.21.15.33:0/633274646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cd4071320 0x7f5cd41e1b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c750 tx=0x7f5cd000cb10 comp rx=0 tx=0).stop 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cda6a8700 1 -- 172.21.15.33:0/633274646 reap_dead start 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cda6a8700 1 -- 172.21.15.33:0/633274646 reap_dead start 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 shutdown_connections 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 >> 172.21.15.33:0/633274646 conn(0x7f5cd406e990 msgr2=0x7f5cd40cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 shutdown_connections 2024-05-08T23:33:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.317+0000 7f5cdc10b700 1 -- 172.21.15.33:0/633274646 wait complete. 2024-05-08T23:33:17.331 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:17.331 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:33:17.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:33:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.527+0000 7f8db8ece700 1 Processor -- start 2024-05-08T23:33:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.527+0000 7f8db8ece700 1 -- start start 2024-05-08T23:33:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.528+0000 7f8db8ece700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8db4152350 0x7f8db4158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.528+0000 7f8db8ece700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db40c85a0 0x7f8db40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.528+0000 7f8db8ece700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db4070450 con 0x7f8db40c38a0 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.528+0000 7f8db8ece700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40c2e90 con 0x7f8db40c85a0 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.528+0000 7f8db8ece700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db4070730 con 0x7f8db4152350 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db40c85a0 0x7f8db40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db40c85a0 0x7f8db40c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51532/0 (socket says 172.21.15.33:51532) 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 -- 172.21.15.33:0/3368772690 learned_addr learned my addr 172.21.15.33:0/3368772690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db259c700 1 --2- 172.21.15.33:0/3368772690 >> v2:172.21.15.42:3300/0 conn(0x7f8db4152350 0x7f8db4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 -- 172.21.15.33:0/3368772690 >> v2:172.21.15.42:3300/0 conn(0x7f8db4152350 msgr2=0x7f8db4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/3368772690 >> v2:172.21.15.42:3300/0 conn(0x7f8db4152350 0x7f8db4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 -- 172.21.15.33:0/3368772690 >> v1:172.21.15.42:6789/0 conn(0x7f8db40c38a0 legacy=0x7f8db40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db159a700 1 -- 172.21.15.33:0/3368772690 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253587988 0 0) 0x7f8db4070450 con 0x7f8db40c38a0 2024-05-08T23:33:17.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 -- 172.21.15.33:0/3368772690 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4070a10 con 0x7f8db40c85a0 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/3368772690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c85a0 0x7f8db40c8a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8db40737d0 tx=0x7f8dac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013fc249fd15872 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db159a700 1 -- 172.21.15.33:0/3368772690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00e070 con 0x7f8db40c85a0 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db159a700 1 -- 172.21.15.33:0/3368772690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00b910 con 0x7f8db40c85a0 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.529+0000 7f8db159a700 1 -- 172.21.15.33:0/3368772690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00bcc0 con 0x7f8db40c85a0 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 -- 172.21.15.33:0/3368772690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c85a0 msgr2=0x7f8db40c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 --2- 172.21.15.33:0/3368772690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c85a0 0x7f8db40c8a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8db40737d0 tx=0x7f8dac00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 -- 172.21.15.33:0/3368772690 shutdown_connections 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 --2- 172.21.15.33:0/3368772690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c85a0 0x7f8db40c8a10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 --2- 172.21.15.33:0/3368772690 >> v2:172.21.15.42:3300/0 conn(0x7f8db4152350 0x7f8db4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 -- 172.21.15.33:0/3368772690 >> 172.21.15.33:0/3368772690 conn(0x7f8db40666c0 msgr2=0x7f8db4066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 -- 172.21.15.33:0/3368772690 shutdown_connections 2024-05-08T23:33:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.530+0000 7f8db8ece700 1 -- 172.21.15.33:0/3368772690 wait complete. 2024-05-08T23:33:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.532+0000 7f8db8ece700 1 Processor -- start 2024-05-08T23:33:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.532+0000 7f8db8ece700 1 -- start start 2024-05-08T23:33:17.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db40c85a0 0x7f8db41f0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db4152350 0x7f8db4157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c58f0 con 0x7f8db40c38a0 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db41f4480 con 0x7f8db4152350 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db406cdd0 con 0x7f8db40c85a0 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51536/0 (socket says 172.21.15.33:51536) 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 -- 172.21.15.33:0/1207879728 learned_addr learned my addr 172.21.15.33:0/1207879728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db4152350 0x7f8db4157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db1d9b700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db40c85a0 0x7f8db41f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db40c85a0 msgr2=0x7f8db41f0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db40c85a0 0x7f8db41f0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db4152350 msgr2=0x7f8db4157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db4152350 0x7f8db4157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac009b30 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8db4152350 0x7f8db4157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db259c700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8da400e870 tx=0x7f8da400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4009590 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400ee70 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db406d300 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.533+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400ca50 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.534+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db41f04d0 con 0x7f8db40c38a0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.534+0000 7f8db8ece700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 0x7f8db41e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.534+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db406b420 con 0x7f8db41e39d0 2024-05-08T23:33:17.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.535+0000 7f8db1d9b700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 0x7f8db41e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.535+0000 7f8db1d9b700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8db40c3cf0 con 0x7f8db41e39d0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.535+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8da400cbb0 con 0x7f8db40c38a0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.535+0000 7f8db1d9b700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 0x7f8db41e2f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8db41edbf0 tx=0x7f8da800ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8d9406acc0 0x7f8d9406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8da4089360 con 0x7f8db40c38a0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8d9406acc0 0x7f8d9406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da80052e0 con 0x7f8db41e39d0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8da8005440 con 0x7f8db41e39d0 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 msgr2=0x7f8db41e2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 0x7f8db41e2f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8db41edbf0 tx=0x7f8da800ea90 comp rx=0 tx=0).stop 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 msgr2=0x7f8db41e2f10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8da2ffd700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41e39d0 0x7f8db41e2f10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.536+0000 7f8db2d9d700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8d9406acc0 0x7f8d9406d180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009fd0 tx=0x7f8dac009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.545+0000 7f8db8ece700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 0x7f8db41e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.545+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4070110 con 0x7f8db41f0080 2024-05-08T23:33:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.545+0000 7f8db1d9b700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 0x7f8db41e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.545+0000 7f8db1d9b700 1 -- 172.21.15.33:0/1207879728 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8db40c3380 con 0x7f8db41f0080 2024-05-08T23:33:17.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8db1d9b700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 0x7f8db41e3450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8da8013000 tx=0x7f8da8001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da801c2a0 con 0x7f8db41f0080 2024-05-08T23:33:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4940 (secure 0 0 0) 0x7f8da8003890 con 0x7f8db41f0080 2024-05-08T23:33:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 msgr2=0x7f8db41e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 0x7f8db41e3450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8da8013000 tx=0x7f8da8001080 comp rx=0 tx=0).stop 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 msgr2=0x7f8db41e3450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8da2ffd700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8db41f0080 0x7f8db41e3450 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8da8013000 tx=0x7f8da8001080 comp rx=0 tx=0).stop 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.547 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:17.549 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:17.550 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.557 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.558 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:17.559 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8d9406acc0 msgr2=0x7f8d9406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.546+0000 7f8db8ece700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8d9406acc0 0x7f8d9406d180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009fd0 tx=0x7f8dac009000 comp rx=0 tx=0).stop 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 msgr2=0x7f8db41f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8da400e870 tx=0x7f8da400ec30 comp rx=0 tx=0).stop 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db2d9d700 1 -- 172.21.15.33:0/1207879728 reap_dead start 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 shutdown_connections 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 --2- 172.21.15.33:0/1207879728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40c38a0 0x7f8db41f3f40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 >> 172.21.15.33:0/1207879728 conn(0x7f8db40666c0 msgr2=0x7f8db4151020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 shutdown_connections 2024-05-08T23:33:17.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.547+0000 7f8db8ece700 1 -- 172.21.15.33:0/1207879728 wait complete. 2024-05-08T23:33:17.562 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:17.562 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:33:17.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:33:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.705+0000 7f76bf760700 1 Processor -- start 2024-05-08T23:33:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.705+0000 7f76bf760700 1 -- start start 2024-05-08T23:33:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bf760700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b80c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bf760700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76b80712f0 0x7f76b815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bf760700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b80c2860 con 0x7f76b806d980 2024-05-08T23:33:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bf760700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b80c3540 con 0x7f76b80712f0 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bf760700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c9600 con 0x7f76b8154540 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76b80712f0 0x7f76b815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f76b80712f0 0x7f76b815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51548/0 (socket says 172.21.15.33:51548) 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 -- 172.21.15.33:0/2130245407 learned_addr learned my addr 172.21.15.33:0/2130245407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bd4fc700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b80c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 -- 172.21.15.33:0/2130245407 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 msgr2=0x7f76b80c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b80c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 -- 172.21.15.33:0/2130245407 >> v1:172.21.15.33:6789/0 conn(0x7f76b806d980 legacy=0x7f76b815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76bccfb700 1 -- 172.21.15.33:0/2130245407 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80c8620 con 0x7f76b80712f0 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.706+0000 7f76affff700 1 -- 172.21.15.33:0/2130245407 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890335407 0 0) 0x7f76b80c2860 con 0x7f76b806d980 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bccfb700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b815d8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098f0 tx=0x7f76a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ed43d2e84af03d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76affff700 1 -- 172.21.15.33:0/2130245407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800b8a0 con 0x7f76b80712f0 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76affff700 1 -- 172.21.15.33:0/2130245407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a800ba00 con 0x7f76b80712f0 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76affff700 1 -- 172.21.15.33:0/2130245407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800bd90 con 0x7f76b80712f0 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 -- 172.21.15.33:0/2130245407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 msgr2=0x7f76b815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b815d8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098f0 tx=0x7f76a8009c00 comp rx=0 tx=0).stop 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 -- 172.21.15.33:0/2130245407 shutdown_connections 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b80c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 --2- 172.21.15.33:0/2130245407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b815d8f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.707+0000 7f76bf760700 1 -- 172.21.15.33:0/2130245407 >> 172.21.15.33:0/2130245407 conn(0x7f76b806ce60 msgr2=0x7f76b806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.708+0000 7f76bf760700 1 -- 172.21.15.33:0/2130245407 shutdown_connections 2024-05-08T23:33:17.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.708+0000 7f76bf760700 1 -- 172.21.15.33:0/2130245407 wait complete. 2024-05-08T23:33:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.709+0000 7f76bf760700 1 Processor -- start 2024-05-08T23:33:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.709+0000 7f76bf760700 1 -- start start 2024-05-08T23:33:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 0x7f76b815b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b81f09a0 con 0x7f76b80712f0 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b806af60 con 0x7f76b806d980 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bf760700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b815bbc0 con 0x7f76b8154540 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51556/0 (socket says 172.21.15.33:51556) 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 -- 172.21.15.33:0/3863465135 learned_addr learned my addr 172.21.15.33:0/3863465135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bd4fc700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 0x7f76b815b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bdcfd700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 -- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 msgr2=0x7f76b806aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b806aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 -- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 msgr2=0x7f76b815b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 0x7f76b815b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 -- 172.21.15.33:0/3863465135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009550 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bd4fc700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 0x7f76b815b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76bccfb700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009c40 tx=0x7f76a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.710+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004380 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.711+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a801f070 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.711+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b815d280 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.711+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80036a0 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.711+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b815d840 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.712+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b8073770 con 0x7f76b80712f0 2024-05-08T23:33:17.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.712+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f76a8021030 con 0x7f76b80712f0 2024-05-08T23:33:17.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.713+0000 7f76adffb700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 0x7f769c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:17.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.713+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f76a808a7f0 con 0x7f76b80712f0 2024-05-08T23:33:17.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.713+0000 7f76bd4fc700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 0x7f769c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:17.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.714+0000 7f76bd4fc700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 0x7f769c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009820 tx=0x7f76b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:17.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:17.718+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f76a8055990 con 0x7f76b80712f0 2024-05-08T23:33:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.043+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f76b806e8d0 con 0x7f76b80712f0 2024-05-08T23:33:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.044+0000 7f76adffb700 1 -- 172.21.15.33:0/3863465135 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f76a8015020 con 0x7f76b80712f0 2024-05-08T23:33:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.046+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 msgr2=0x7f769c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.046+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 0x7f769c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009820 tx=0x7f76b4008040 comp rx=0 tx=0).stop 2024-05-08T23:33:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.046+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 msgr2=0x7f76b81f0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.046+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009c40 tx=0x7f76a8004750 comp rx=0 tx=0).stop 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 shutdown_connections 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f769c06ac90 0x7f769c06d150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f76b8154540 0x7f76b806aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76b806d980 0x7f76b815b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 --2- 172.21.15.33:0/3863465135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76b80712f0 0x7f76b81f0460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 >> 172.21.15.33:0/3863465135 conn(0x7f76b806ce60 msgr2=0x7f76b814ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 shutdown_connections 2024-05-08T23:33:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:18.047+0000 7f76bf760700 1 -- 172.21.15.33:0/3863465135 wait complete. 2024-05-08T23:33:18.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:33:28.061 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:33:28.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.198+0000 7fc898f9b700 1 Processor -- start 2024-05-08T23:33:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.198+0000 7fc898f9b700 1 -- start start 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.199+0000 7fc898f9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8940c8e50 0x7fc89415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.199+0000 7fc898f9b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89406baa0 con 0x7fc89415e280 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.199+0000 7fc898f9b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89406bcf0 con 0x7fc894154520 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.199+0000 7fc898f9b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89415e730 con 0x7fc8940c8e50 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc891d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8940c8e50 0x7fc89415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc892d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc89415e280 0x7fc894070140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38922/0 (socket says 172.21.15.33:38922) 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc892d9d700 1 -- 172.21.15.33:0/2240934500 learned_addr learned my addr 172.21.15.33:0/2240934500 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc89159a700 1 -- 172.21.15.33:0/2240934500 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638186333 0 0) 0x7fc89406baa0 con 0x7fc89415e280 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc89159a700 1 -- 172.21.15.33:0/2240934500 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc87c0035b0 con 0x7fc89415e280 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc891d9b700 1 -- 172.21.15.33:0/2240934500 >> v1:172.21.15.42:6789/0 conn(0x7fc894154520 legacy=0x7fc8940cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc891d9b700 1 -- 172.21.15.33:0/2240934500 >> v1:172.21.15.33:6789/0 conn(0x7fc89415e280 legacy=0x7fc894070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc891d9b700 1 -- 172.21.15.33:0/2240934500 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89406bfd0 con 0x7fc8940c8e50 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc891d9b700 1 --2- 172.21.15.33:0/2240934500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8940c8e50 0x7fc89415dd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc888009840 tx=0x7fc888009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b3b9d2080b8c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc89159a700 1 -- 172.21.15.33:0/2240934500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888011830 con 0x7fc8940c8e50 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc89159a700 1 -- 172.21.15.33:0/2240934500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888011990 con 0x7fc8940c8e50 2024-05-08T23:33:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.200+0000 7fc89159a700 1 -- 172.21.15.33:0/2240934500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888011d00 con 0x7fc8940c8e50 2024-05-08T23:33:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 -- 172.21.15.33:0/2240934500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8940c8e50 msgr2=0x7fc89415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 --2- 172.21.15.33:0/2240934500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8940c8e50 0x7fc89415dd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc888009840 tx=0x7fc888009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 -- 172.21.15.33:0/2240934500 shutdown_connections 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 --2- 172.21.15.33:0/2240934500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8940c8e50 0x7fc89415dd40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 -- 172.21.15.33:0/2240934500 >> 172.21.15.33:0/2240934500 conn(0x7fc894072230 msgr2=0x7fc894072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 -- 172.21.15.33:0/2240934500 shutdown_connections 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.201+0000 7fc898f9b700 1 -- 172.21.15.33:0/2240934500 wait complete. 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.203+0000 7fc898f9b700 1 Processor -- start 2024-05-08T23:33:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.203+0000 7fc898f9b700 1 -- start start 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8940c8e50 0x7fc8941484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc894154520 0x7fc8941474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c8420 con 0x7fc89415e280 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894159fe0 con 0x7fc8940c8e50 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc898f9b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c8630 con 0x7fc894154520 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38770/0 (socket says 172.21.15.33:38770) 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 -- 172.21.15.33:0/1837551796 learned_addr learned my addr 172.21.15.33:0/1837551796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc89259c700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8940c8e50 0x7fc8941484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc891d9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc894154520 0x7fc8941474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 -- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc894154520 msgr2=0x7fc8941474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc894154520 0x7fc8941474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 -- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8940c8e50 msgr2=0x7fc8941484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8940c8e50 0x7fc8941484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 -- 172.21.15.33:0/1837551796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880094a0 con 0x7fc89415e280 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.204+0000 7fc892d9d700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00caa0 tx=0x7fc88c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.205+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00a9e0 con 0x7fc89415e280 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.205+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c00ab40 con 0x7fc89415e280 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.205+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012400 con 0x7fc89415e280 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.205+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940c8910 con 0x7fc89415e280 2024-05-08T23:33:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.205+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89415add0 con 0x7fc89415e280 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.206+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89415afc0 con 0x7fc89415e280 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.207+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc88c01b3b0 con 0x7fc89415e280 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.207+0000 7fc882ffd700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 0x7fc87406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.207+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fc88c088fb0 con 0x7fc89415e280 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.207+0000 7fc89259c700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 0x7fc87406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.208+0000 7fc89259c700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 0x7fc87406d180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8840018a0 tx=0x7fc884009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.212+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc88c0bc900 con 0x7fc89415e280 2024-05-08T23:33:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.567+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc894075460 con 0x7fc89415e280 2024-05-08T23:33:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.567+0000 7fc882ffd700 1 -- 172.21.15.33:0/1837551796 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fc88c017620 con 0x7fc89415e280 2024-05-08T23:33:28.567 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 msgr2=0x7fc87406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 0x7fc87406d180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc8840018a0 tx=0x7fc884009040 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 msgr2=0x7fc8941479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00caa0 tx=0x7fc88c00cdb0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 shutdown_connections 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc87406acc0 0x7fc87406d180 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc89415e280 0x7fc8941479f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8940c8e50 0x7fc8941484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 --2- 172.21.15.33:0/1837551796 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc894154520 0x7fc8941474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 >> 172.21.15.33:0/1837551796 conn(0x7fc894072230 msgr2=0x7fc8940c2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 shutdown_connections 2024-05-08T23:33:28.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.570+0000 7fc898f9b700 1 -- 172.21.15.33:0/1837551796 wait complete. 2024-05-08T23:33:28.584 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:33:28.584 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:33:28.584 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:33:28.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:33:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.724+0000 7f90f8163700 1 Processor -- start 2024-05-08T23:33:28.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.725+0000 7f90f8163700 1 -- start start 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.725+0000 7f90f8163700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f0154970 0x7f90f006ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f8163700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f8163700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f015b350 con 0x7f90f006b280 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f8163700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f015e8a0 con 0x7f90f0154970 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f8163700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015a5d0 con 0x7f90f00caad0 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f0154970 0x7f90f006ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90f0154970 0x7f90f006ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38786/0 (socket says 172.21.15.33:38786) 2024-05-08T23:33:28.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 -- 172.21.15.33:0/3321002705 learned_addr learned my addr 172.21.15.33:0/3321002705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f56fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f90f006b280 0x7f90f00cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38936/0 (socket says 172.21.15.33:38936) 2024-05-08T23:33:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f6700700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 -- 172.21.15.33:0/3321002705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 msgr2=0x7f90f015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f015d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 -- 172.21.15.33:0/3321002705 >> v1:172.21.15.33:6789/0 conn(0x7f90f006b280 legacy=0x7f90f00cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 -- 172.21.15.33:0/3321002705 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f015e5c0 con 0x7f90f0154970 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f6700700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.726+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f006ad40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098f0 tx=0x7f90ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dead52c50f7c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f4efd700 1 -- 172.21.15.33:0/3321002705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00b8a0 con 0x7f90f0154970 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f4efd700 1 -- 172.21.15.33:0/3321002705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00ba00 con 0x7f90f0154970 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f4efd700 1 -- 172.21.15.33:0/3321002705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00bd90 con 0x7f90f0154970 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 -- 172.21.15.33:0/3321002705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 msgr2=0x7f90f006ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f006ad40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098f0 tx=0x7f90ec009c00 comp rx=0 tx=0).stop 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 -- 172.21.15.33:0/3321002705 shutdown_connections 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 --2- 172.21.15.33:0/3321002705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f006ad40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 -- 172.21.15.33:0/3321002705 >> 172.21.15.33:0/3321002705 conn(0x7f90f006cd80 msgr2=0x7f90f006d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 -- 172.21.15.33:0/3321002705 shutdown_connections 2024-05-08T23:33:28.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.727+0000 7f90f8163700 1 -- 172.21.15.33:0/3321002705 wait complete. 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 Processor -- start 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 -- start start 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f006b280 0x7f90f014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f0159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01f0f10 con 0x7f90f0154970 2024-05-08T23:33:28.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01f0390 con 0x7f90f006b280 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.729+0000 7f90f8163700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01f0670 con 0x7f90f00caad0 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f0159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f0159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38796/0 (socket says 172.21.15.33:38796) 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 learned_addr learned my addr 172.21.15.33:0/3458400753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f56fe700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f006b280 0x7f90f014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 msgr2=0x7f90f01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f90f00caad0 0x7f90f01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f006b280 msgr2=0x7f90f014e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f006b280 0x7f90f014e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec009550 con 0x7f90f0154970 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90f006b280 0x7f90f014e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f6700700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f0159830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007ef0 tx=0x7f90e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800de60 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e800f040 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f01f0950 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003c60 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.730+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f015b6c0 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.731+0000 7f90f8163700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 0x7f90f015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.731+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f0074690 con 0x7f90f00c6f30 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.731+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 0x7f90f015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.732+0000 7f90f5eff700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90f015ae30 con 0x7f90f00c6f30 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.732+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 0x7f90f015bbe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098f0 tx=0x7f90ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.732+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f90e8004030 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.732+0000 7f90e67fc700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f90d406ac90 0x7f90d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f90e8089590 con 0x7f90f0154970 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90f56fe700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f90d406ac90 0x7f90d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec004400 con 0x7f90f00c6f30 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f90ec004560 con 0x7f90f00c6f30 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 msgr2=0x7f90f015bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 0x7f90f015bbe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098f0 tx=0x7f90ec004750 comp rx=0 tx=0).stop 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 msgr2=0x7f90f015bbe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90e67fc700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f00c6f30 0x7f90f015bbe0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0098f0 tx=0x7f90ec004750 comp rx=0 tx=0).stop 2024-05-08T23:33:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.733+0000 7f90f56fe700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f90d406ac90 0x7f90d406d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f90f00cbca0 tx=0x7f90e0007d10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.741+0000 7f90f8163700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 0x7f90f01f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.741+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01f0ae0 con 0x7f90f015c120 2024-05-08T23:33:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.741+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 0x7f90f01f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.741+0000 7f90f5eff700 1 -- 172.21.15.33:0/3458400753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90f015ab50 con 0x7f90f015c120 2024-05-08T23:33:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.741+0000 7f90f5eff700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 0x7f90f01f3530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009c40 tx=0x7f90ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec01abe0 con 0x7f90f015c120 2024-05-08T23:33:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4970 (secure 0 0 0) 0x7f90ec02f070 con 0x7f90f015c120 2024-05-08T23:33:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 msgr2=0x7f90f01f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 0x7f90f01f3530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009c40 tx=0x7f90ec011000 comp rx=0 tx=0).stop 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 msgr2=0x7f90f01f3530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90e67fc700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f015c120 0x7f90f01f3530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:28.743 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 12, 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:28.758 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:28.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f90d406ac90 msgr2=0x7f90d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f90d406ac90 0x7f90d406d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f90f00cbca0 tx=0x7f90e0007d10 comp rx=0 tx=0).stop 2024-05-08T23:33:28.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 msgr2=0x7f90f0159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 --2- 172.21.15.33:0/3458400753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f0154970 0x7f90f0159830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007ef0 tx=0x7f90e800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 reap_dead start 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f6700700 1 -- 172.21.15.33:0/3458400753 reap_dead start 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 shutdown_connections 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 >> 172.21.15.33:0/3458400753 conn(0x7f90f006cd80 msgr2=0x7f90f0152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 shutdown_connections 2024-05-08T23:33:28.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.742+0000 7f90f8163700 1 -- 172.21.15.33:0/3458400753 wait complete. 2024-05-08T23:33:28.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:28.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:33:28.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:33:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.905+0000 7f42dc5cb700 1 Processor -- start 2024-05-08T23:33:28.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.905+0000 7f42dc5cb700 1 -- start start 2024-05-08T23:33:28.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.906+0000 7f42dc5cb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 0x7f42d40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dc5cb700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f42d40ce150 0x7f42d40c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dc5cb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d415e6e0 con 0x7f42d4153fd0 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dc5cb700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40c9090 con 0x7f42d40ce150 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dc5cb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c3b60 con 0x7f42d40cdd70 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dab68700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f42d4153fd0 0x7f42d4154460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38940/0 (socket says 172.21.15.33:38940) 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42dab68700 1 -- 172.21.15.33:0/1963317999 learned_addr learned my addr 172.21.15.33:0/1963317999 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9b66700 1 --2- 172.21.15.33:0/1963317999 >> v2:172.21.15.42:3300/0 conn(0x7f42d40ce150 0x7f42d40c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42da367700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 0x7f42d40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9365700 1 -- 172.21.15.33:0/1963317999 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966599106 0 0) 0x7f42d415e6e0 con 0x7f42d4153fd0 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9365700 1 -- 172.21.15.33:0/1963317999 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c00035b0 con 0x7f42d4153fd0 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9b66700 1 -- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 msgr2=0x7f42d40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9b66700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 0x7f42d40c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9b66700 1 -- 172.21.15.33:0/1963317999 >> v1:172.21.15.33:6789/0 conn(0x7f42d4153fd0 legacy=0x7f42d4154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42d9b66700 1 -- 172.21.15.33:0/1963317999 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d415db60 con 0x7f42d40ce150 2024-05-08T23:33:28.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.907+0000 7f42da367700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 0x7f42d40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:28.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42d9b66700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40ce150 0x7f42d40c9a60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009840 tx=0x7f42c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=965149788c1a4c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42d9365700 1 -- 172.21.15.33:0/1963317999 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011830 con 0x7f42d40ce150 2024-05-08T23:33:28.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42d9365700 1 -- 172.21.15.33:0/1963317999 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4011990 con 0x7f42d40ce150 2024-05-08T23:33:28.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42d9365700 1 -- 172.21.15.33:0/1963317999 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011d20 con 0x7f42d40ce150 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42dc5cb700 1 -- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40ce150 msgr2=0x7f42d40c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.908+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40ce150 0x7f42d40c9a60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009840 tx=0x7f42c4009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 -- 172.21.15.33:0/1963317999 shutdown_connections 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d40cdd70 0x7f42d40c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/1963317999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40ce150 0x7f42d40c9a60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 -- 172.21.15.33:0/1963317999 >> 172.21.15.33:0/1963317999 conn(0x7f42d40666c0 msgr2=0x7f42d4066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 -- 172.21.15.33:0/1963317999 shutdown_connections 2024-05-08T23:33:28.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.909+0000 7f42dc5cb700 1 -- 172.21.15.33:0/1963317999 wait complete. 2024-05-08T23:33:28.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.910+0000 7f42dc5cb700 1 Processor -- start 2024-05-08T23:33:28.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.910+0000 7f42dc5cb700 1 -- start start 2024-05-08T23:33:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cdd70 0x7f42d40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 0x7f42d40c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d4153fd0 0x7f42d40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c9730 con 0x7f42d40ce150 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40cb4f0 con 0x7f42d40cdd70 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dc5cb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40cb230 con 0x7f42d4153fd0 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42d9b66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 0x7f42d40c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42d9b66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 0x7f42d40c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38818/0 (socket says 172.21.15.33:38818) 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42d9b66700 1 -- 172.21.15.33:0/3542595771 learned_addr learned my addr 172.21.15.33:0/3542595771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cdd70 0x7f42d40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.911+0000 7f42dab68700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d4153fd0 0x7f42d40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d4153fd0 msgr2=0x7f42d40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42d4153fd0 0x7f42d40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cdd70 msgr2=0x7f42d40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cdd70 0x7f42d40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00094a0 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cdd70 0x7f42d40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42d9b66700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 0x7f42d40c9bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f42c40090b0 tx=0x7f42c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4004590 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c401f070 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c40094a0 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40038e0 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.912+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d415d2a0 con 0x7f42d40ce150 2024-05-08T23:33:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.913+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 0x7f42d4143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.913+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d406b420 con 0x7f42d414e1a0 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.913+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 0x7f42d4143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.914+0000 7f42da367700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42d40c9520 con 0x7f42d414e1a0 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.914+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 0x7f42d4143560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42d0011f00 tx=0x7f42d000f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.914+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004030 con 0x7f42d414e1a0 2024-05-08T23:33:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.914+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f42c4003a40 con 0x7f42d40ce150 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f42b806ad30 0x7f42b806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f42c4021070 con 0x7f42d40ce150 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42dab68700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f42b806ad30 0x7f42b806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f42d0004190 con 0x7f42d414e1a0 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 msgr2=0x7f42d4143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 0x7f42d4143560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f42d0011f00 tx=0x7f42d000f520 comp rx=0 tx=0).stop 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 msgr2=0x7f42d4143560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42caffd700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d414e1a0 0x7f42d4143560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.915+0000 7f42dab68700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f42b806ad30 0x7f42b806d1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001560 tx=0x7f42cc001970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.924+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 0x7f42d4143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.924+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d415e400 con 0x7f42d40cab60 2024-05-08T23:33:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.924+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 0x7f42d4143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.924+0000 7f42da367700 1 -- 172.21.15.33:0/3542595771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42d415e120 con 0x7f42d40cab60 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.924+0000 7f42da367700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 0x7f42d4143aa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42d001b000 tx=0x7f42d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d001ea50 con 0x7f42d40cab60 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4458 (secure 0 0 0) 0x7f42d0029400 con 0x7f42d40cab60 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 msgr2=0x7f42d4143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 0x7f42d4143aa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42d001b000 tx=0x7f42d0004750 comp rx=0 tx=0).stop 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 msgr2=0x7f42d4143aa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42caffd700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d40cab60 0x7f42d4143aa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 12, 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:28.937 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f42b806ad30 msgr2=0x7f42b806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f42b806ad30 0x7f42b806d1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001560 tx=0x7f42cc001970 comp rx=0 tx=0).stop 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 msgr2=0x7f42d40c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 --2- 172.21.15.33:0/3542595771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d40ce150 0x7f42d40c9bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f42c40090b0 tx=0x7f42c4004750 comp rx=0 tx=0).stop 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dab68700 1 -- 172.21.15.33:0/3542595771 reap_dead start 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dab68700 1 -- 172.21.15.33:0/3542595771 reap_dead start 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 shutdown_connections 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 >> 172.21.15.33:0/3542595771 conn(0x7f42d40666c0 msgr2=0x7f42d414fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:28.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.925+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 shutdown_connections 2024-05-08T23:33:28.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:28.926+0000 7f42dc5cb700 1 -- 172.21.15.33:0/3542595771 wait complete. 2024-05-08T23:33:28.940 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:28.940 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:33:28.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:33:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 Processor -- start 2024-05-08T23:33:29.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 -- start start 2024-05-08T23:33:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91940caad0 0x7f91940cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 0x7f919415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919415b350 con 0x7f9194154970 2024-05-08T23:33:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.078+0000 7f919a67f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919415e8a0 con 0x7f91940caad0 2024-05-08T23:33:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f919a67f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919415a5d0 con 0x7f91940cbbe0 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9193fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9194154970 0x7f919406ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38948/0 (socket says 172.21.15.33:38948) 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9193fff700 1 -- 172.21.15.33:0/3430599082 learned_addr learned my addr 172.21.15.33:0/3430599082 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 --2- 172.21.15.33:0/3430599082 >> v2:172.21.15.42:3300/0 conn(0x7f91940caad0 0x7f91940cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9198c1c700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 0x7f919415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9192ffd700 1 -- 172.21.15.33:0/3430599082 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567966428 0 0) 0x7f919415b350 con 0x7f9194154970 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9192ffd700 1 -- 172.21.15.33:0/3430599082 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c0035b0 con 0x7f9194154970 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 -- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 msgr2=0x7f919415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 0x7f919415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 -- 172.21.15.33:0/3430599082 >> v1:172.21.15.33:6789/0 conn(0x7f9194154970 legacy=0x7f919406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 -- 172.21.15.33:0/3430599082 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919415e5c0 con 0x7f91940caad0 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f9198c1c700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 0x7f919415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.079+0000 7f91937fe700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940caad0 0x7f91940cb6a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9188009840 tx=0x7f9188009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f3091869e8d6dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f9192ffd700 1 -- 172.21.15.33:0/3430599082 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188011830 con 0x7f91940caad0 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f9192ffd700 1 -- 172.21.15.33:0/3430599082 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9188011990 con 0x7f91940caad0 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f9192ffd700 1 -- 172.21.15.33:0/3430599082 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188011d20 con 0x7f91940caad0 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 -- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940caad0 msgr2=0x7f91940cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940caad0 0x7f91940cb6a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9188009840 tx=0x7f9188009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 -- 172.21.15.33:0/3430599082 shutdown_connections 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940cbbe0 0x7f919415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 --2- 172.21.15.33:0/3430599082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940caad0 0x7f91940cb6a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 -- 172.21.15.33:0/3430599082 >> 172.21.15.33:0/3430599082 conn(0x7f919406cd80 msgr2=0x7f919406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 -- 172.21.15.33:0/3430599082 shutdown_connections 2024-05-08T23:33:29.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.080+0000 7f919a67f700 1 -- 172.21.15.33:0/3430599082 wait complete. 2024-05-08T23:33:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.082+0000 7f919a67f700 1 Processor -- start 2024-05-08T23:33:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.082+0000 7f919a67f700 1 -- start start 2024-05-08T23:33:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940cbbe0 0x7f91940c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9194154970 0x7f91940c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940c7c30 con 0x7f91940caad0 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941f10e0 con 0x7f91940cbbe0 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f919a67f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940c7060 con 0x7f9194154970 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38834/0 (socket says 172.21.15.33:38834) 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 -- 172.21.15.33:0/879994236 learned_addr learned my addr 172.21.15.33:0/879994236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f91937fe700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940cbbe0 0x7f91940c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9198c1c700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9194154970 0x7f91940c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9194154970 msgr2=0x7f91940c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9194154970 0x7f91940c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940cbbe0 msgr2=0x7f91940c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91940cbbe0 0x7f91940c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c009a70 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.083+0000 7f9193fff700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9184001eb0 tx=0x7f918400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400cea0 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184005d70 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880094a0 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91840072f0 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940c75e0 con 0x7f91940caad0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f919a67f700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 0x7f91941f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.084+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9194074690 con 0x7f91940c77d0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.085+0000 7f91937fe700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 0x7f91941f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.085+0000 7f91937fe700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91940c9ff0 con 0x7f91940c77d0 2024-05-08T23:33:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.085+0000 7f91937fe700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 0x7f91941f36e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9188009840 tx=0x7f9188004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.085+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188020070 con 0x7f91940c77d0 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.086+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9188004380 con 0x7f91940c77d0 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.086+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 msgr2=0x7f91941f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.086+0000 7f9190ff9700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 0x7f91941f36e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9188009840 tx=0x7f9188004750 comp rx=0 tx=0).stop 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.086+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 msgr2=0x7f91941f36e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.086+0000 7f9190ff9700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91940c77d0 0x7f91941f36e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.087+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9184007450 con 0x7f91940caad0 2024-05-08T23:33:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.087+0000 7f9190ff9700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f918006a9a0 0x7f918006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.087+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f91840887e0 con 0x7f91940caad0 2024-05-08T23:33:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.087+0000 7f91937fe700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f918006a9a0 0x7f918006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.088+0000 7f91937fe700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f918006a9a0 0x7f918006ce60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91940cf540 tx=0x7f918801d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.094+0000 7f919a67f700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 0x7f91940c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.094+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940c6320 con 0x7f91941f0bf0 2024-05-08T23:33:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.095+0000 7f9198c1c700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 0x7f91940c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.095+0000 7f9198c1c700 1 -- 172.21.15.33:0/879994236 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91940b55c0 con 0x7f91941f0bf0 2024-05-08T23:33:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.095+0000 7f9198c1c700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 0x7f91940c49a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f918c00f3e0 tx=0x7f918c00f7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c00e040 con 0x7f91941f0bf0 2024-05-08T23:33:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4941 (secure 0 0 0) 0x7f918c00f040 con 0x7f91941f0bf0 2024-05-08T23:33:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 msgr2=0x7f91940c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 0x7f91940c49a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f918c00f3e0 tx=0x7f918c00f7a0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 msgr2=0x7f91940c49a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.096+0000 7f9190ff9700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91941f0bf0 0x7f91940c49a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 16, 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 12, 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:29.110 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f918006a9a0 msgr2=0x7f918006ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f918006a9a0 0x7f918006ce60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91940cf540 tx=0x7f918801d000 comp rx=0 tx=0).stop 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 msgr2=0x7f919415cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9184001eb0 tx=0x7f918400c7b0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f9198c1c700 1 -- 172.21.15.33:0/879994236 reap_dead start 2024-05-08T23:33:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 shutdown_connections 2024-05-08T23:33:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 --2- 172.21.15.33:0/879994236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91940caad0 0x7f919415cea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 >> 172.21.15.33:0/879994236 conn(0x7f919406cd80 msgr2=0x7f919414ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 shutdown_connections 2024-05-08T23:33:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.097+0000 7f919a67f700 1 -- 172.21.15.33:0/879994236 wait complete. 2024-05-08T23:33:29.112 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:33:29.112 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-05-08T23:33:29.113 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:33:29.113 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-08T23:33:29.113 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T23:33:29.113 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T23:33:29.113 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:33:29.127 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T23:33:29.127 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:33:29.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.281+0000 7ff53e3b9700 1 Processor -- start 2024-05-08T23:33:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.281+0000 7ff53e3b9700 1 -- start start 2024-05-08T23:33:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.282+0000 7ff53e3b9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff53806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff53e3b9700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380711c0 con 0x7ff5380c9220 2024-05-08T23:33:29.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff53e3b9700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff538071710 con 0x7ff5380b1ad0 2024-05-08T23:33:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff53e3b9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53806e320 con 0x7ff53815b360 2024-05-08T23:33:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff537fff700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 msgr2=0x7ff53806a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff537fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff53806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff5377fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff5380c9220 0x7ff5380729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38960/0 (socket says 172.21.15.33:38960) 2024-05-08T23:33:29.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff5377fe700 1 -- 172.21.15.33:0/3221219392 learned_addr learned my addr 172.21.15.33:0/3221219392 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344180417 0 0) 0x7ff5380711c0 con 0x7ff5380c9220 2024-05-08T23:33:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5200035b0 con 0x7ff5380c9220 2024-05-08T23:33:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809536552 0 0) 0x7ff538071710 con 0x7ff5380b1ad0 2024-05-08T23:33:29.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.283+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5380711c0 con 0x7ff5380b1ad0 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917180150 0 0) 0x7ff5380711c0 con 0x7ff5380b1ad0 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 msgr2=0x7ff53806a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff53806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 >> v1:172.21.15.33:6789/0 conn(0x7ff5380c9220 legacy=0x7ff5380729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538071430 con 0x7ff5380b1ad0 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7ff5300019d0 con 0x7ff5380b1ad0 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 >> v1:172.21.15.42:6789/0 conn(0x7ff5380b1ad0 legacy=0x7ff538164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff520004c20 0x7ff520005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff5200037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff520003d30 0x7ff5200041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c001230 con 0x7ff520004c20 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53806e320 con 0x7ff520003d30 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52000bbf0 con 0x7ff53815b360 2024-05-08T23:33:29.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff520004c20 0x7ff520005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff5377fe700 1 -- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 msgr2=0x7ff5200037f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff5377fe700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff5200037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff53c956700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff520003d30 0x7ff5200041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 -- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 msgr2=0x7ff5200037f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff53815b360 0x7ff5200037f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 -- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff520003d30 msgr2=0x7ff5200041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff520003d30 0x7ff5200041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 -- 172.21.15.33:0/3221219392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380711c0 con 0x7ff520004c20 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.284+0000 7ff537fff700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff520004c20 0x7ff520005000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff528009e60 tx=0x7ff520005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680b60f665e9ddb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.285+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280098d0 con 0x7ff520004c20 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.285+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800bd20 con 0x7ff520004c20 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.285+0000 7ff536ffd700 1 -- 172.21.15.33:0/3221219392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528003c80 con 0x7ff520004c20 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.285+0000 7ff53e3b9700 1 -- 172.21.15.33:0/3221219392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff520004c20 msgr2=0x7ff520005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.285+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/3221219392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff520004c20 0x7ff520005000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff528009e60 tx=0x7ff520005880 comp rx=0 tx=0).stop 2024-05-08T23:33:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.286+0000 7ff53c956700 1 -- 172.21.15.33:0/3221219392 reap_dead start 2024-05-08T23:33:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.286+0000 7ff53e3b9700 1 -- 172.21.15.33:0/3221219392 shutdown_connections 2024-05-08T23:33:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.286+0000 7ff53e3b9700 1 -- 172.21.15.33:0/3221219392 >> 172.21.15.33:0/3221219392 conn(0x7ff53806d2b0 msgr2=0x7ff53806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.286+0000 7ff53e3b9700 1 -- 172.21.15.33:0/3221219392 shutdown_connections 2024-05-08T23:33:29.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.286+0000 7ff53e3b9700 1 -- 172.21.15.33:0/3221219392 wait complete. 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.287+0000 7ff53e3b9700 1 Processor -- start 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.288+0000 7ff53e3b9700 1 -- start start 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 0x7ff5380af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 0x7ff538153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538153ad0 con 0x7ff538154e70 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538152910 con 0x7ff538155830 2024-05-08T23:33:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53e3b9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff538152b70 con 0x7ff538155c10 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff5377fe700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 msgr2=0x7ff5380af8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff5377fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 0x7ff5380af8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53c956700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 0x7ff538153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:34736/0 (socket says 172.21.15.33:34736) 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 -- 172.21.15.33:0/1139514416 learned_addr learned my addr 172.21.15.33:0/1139514416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 -- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 msgr2=0x7ff5380af8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 0x7ff5380af8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 -- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 msgr2=0x7ff538153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 0x7ff538153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 -- 172.21.15.33:0/1139514416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538152e50 con 0x7ff538155830 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff53c956700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 0x7ff538153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.289+0000 7ff537fff700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff528009fd0 tx=0x7ff528005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.290+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52801c040 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.290+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800be90 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.290+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff538153130 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.290+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381536f0 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.290+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528005520 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.291+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5180052f0 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.291+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff528005680 con 0x7ff538155830 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.292+0000 7ff534ff9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 0x7ff52406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.292+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff528008050 con 0x7ff538155830 2024-05-08T23:33:29.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.292+0000 7ff5377fe700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 0x7ff52406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:29.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.293+0000 7ff5377fe700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 0x7ff52406d130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0011e0 tx=0x7ff52c001530 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.297+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff5280bd900 con 0x7ff538155830 2024-05-08T23:33:29.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.698+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff518005b20 con 0x7ff538155830 2024-05-08T23:33:29.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.699+0000 7ff534ff9700 1 -- 172.21.15.33:0/1139514416 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7ff528055300 con 0x7ff538155830 2024-05-08T23:33:29.699 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 msgr2=0x7ff52406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 0x7ff52406d130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0011e0 tx=0x7ff52c001530 comp rx=0 tx=0).stop 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 msgr2=0x7ff5381efe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff528009fd0 tx=0x7ff528005b10 comp rx=0 tx=0).stop 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 shutdown_connections 2024-05-08T23:33:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff52406ac70 0x7ff52406d130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff538154e70 0x7ff538153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff538155830 0x7ff5381efe30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 --2- 172.21.15.33:0/1139514416 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff538155c10 0x7ff5380af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.702+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 >> 172.21.15.33:0/1139514416 conn(0x7ff53806d2b0 msgr2=0x7ff5380c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.703+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 shutdown_connections 2024-05-08T23:33:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:29.703+0000 7ff53e3b9700 1 -- 172.21.15.33:0/1139514416 wait complete. 2024-05-08T23:33:32.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.873+0000 7f87ba292700 1 Processor -- start 2024-05-08T23:33:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.873+0000 7f87ba292700 1 -- start start 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87ba292700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 0x7f87b415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87ba292700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87b40d00b0 0x7f87b40d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87ba292700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40c5c20 con 0x7f87b40c4c70 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87ba292700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40b30b0 con 0x7f87b40d00b0 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87ba292700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4154b50 con 0x7f87b4154770 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87b40d00b0 0x7f87b40d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b3fff700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 msgr2=0x7f87b415ae50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87b40d00b0 0x7f87b40d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38862/0 (socket says 172.21.15.33:38862) 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 -- 172.21.15.33:0/479697458 learned_addr learned my addr 172.21.15.33:0/479697458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b3fff700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 0x7f87b415ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 -- 172.21.15.33:0/479697458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 msgr2=0x7f87b415ae50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 0x7f87b415ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 -- 172.21.15.33:0/479697458 >> v1:172.21.15.42:6789/0 conn(0x7f87b40c4c70 legacy=0x7f87b40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.875+0000 7f87b882f700 1 -- 172.21.15.33:0/479697458 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40caed0 con 0x7f87b40d00b0 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87b882f700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b40d00b0 0x7f87b40d0540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009e50 tx=0x7f87ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936181767f717279 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87b2ffd700 1 -- 172.21.15.33:0/479697458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00e070 con 0x7f87b40d00b0 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87b2ffd700 1 -- 172.21.15.33:0/479697458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac00b9c0 con 0x7f87b40d00b0 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87b2ffd700 1 -- 172.21.15.33:0/479697458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00bd70 con 0x7f87b40d00b0 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87ba292700 1 -- 172.21.15.33:0/479697458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b40d00b0 msgr2=0x7f87b40d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:32.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87ba292700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b40d00b0 0x7f87b40d0540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009e50 tx=0x7f87ac00b330 comp rx=0 tx=0).stop 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87ba292700 1 -- 172.21.15.33:0/479697458 shutdown_connections 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.876+0000 7f87ba292700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b40d00b0 0x7f87b40d0540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.877+0000 7f87ba292700 1 --2- 172.21.15.33:0/479697458 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b4154770 0x7f87b415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.877+0000 7f87ba292700 1 -- 172.21.15.33:0/479697458 >> 172.21.15.33:0/479697458 conn(0x7f87b4066ae0 msgr2=0x7f87b4066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.877+0000 7f87ba292700 1 -- 172.21.15.33:0/479697458 shutdown_connections 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.877+0000 7f87ba292700 1 -- 172.21.15.33:0/479697458 wait complete. 2024-05-08T23:33:32.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.878+0000 7f87ba292700 1 Processor -- start 2024-05-08T23:33:32.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.878+0000 7f87ba292700 1 -- start start 2024-05-08T23:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40c4c70 0x7f87b41f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 0x7f87b41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4156170 con 0x7f87b4154770 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b414a150 con 0x7f87b40c4c70 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87ba292700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4158c80 con 0x7f87b40d00b0 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b37fe700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 msgr2=0x7f87b41496a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b37fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 0x7f87b41496a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40c4c70 0x7f87b41f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38864/0 (socket says 172.21.15.33:38864) 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 -- 172.21.15.33:0/3237934020 learned_addr learned my addr 172.21.15.33:0/3237934020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 -- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 msgr2=0x7f87b41496a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 0x7f87b41496a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 -- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40c4c70 msgr2=0x7f87b41f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40c4c70 0x7f87b41f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 -- 172.21.15.33:0/3237934020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009b30 con 0x7f87b4154770 2024-05-08T23:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.880+0000 7f87b882f700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87b4073780 tx=0x7f87ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.881+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac004830 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.881+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac0092e0 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.881+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b4156ee0 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.881+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01c680 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.881+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b41586c0 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.882+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4157070 con 0x7f87b4154770 2024-05-08T23:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.883+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f87ac01c7e0 con 0x7f87b4154770 2024-05-08T23:33:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.883+0000 7f87b0ff9700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 0x7f87a006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.884+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f87ac08b0a0 con 0x7f87b4154770 2024-05-08T23:33:32.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.884+0000 7f87b3fff700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 0x7f87a006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.884+0000 7f87b3fff700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 0x7f87a006d100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009450 tx=0x7f87a40093e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:32.889+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f87ac056240 con 0x7f87b4154770 2024-05-08T23:33:33.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.325+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f87b414ee60 con 0x7f87b4154770 2024-05-08T23:33:33.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.325+0000 7f87b0ff9700 1 -- 172.21.15.33:0/3237934020 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f87ac059860 con 0x7f87b4154770 2024-05-08T23:33:33.325 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:33.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 msgr2=0x7f87a006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 0x7f87a006d100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009450 tx=0x7f87a40093e0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 msgr2=0x7f87b4154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87b4073780 tx=0x7f87ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 shutdown_connections 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f87a006ac40 0x7f87a006d100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b4154770 0x7f87b4154d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40c4c70 0x7f87b41f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 --2- 172.21.15.33:0/3237934020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f87b40d00b0 0x7f87b41496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 >> 172.21.15.33:0/3237934020 conn(0x7f87b4066ae0 msgr2=0x7f87b4153740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 shutdown_connections 2024-05-08T23:33:33.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:33.329+0000 7f87ba292700 1 -- 172.21.15.33:0/3237934020 wait complete. 2024-05-08T23:33:36.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.508+0000 7fe9922ae700 1 Processor -- start 2024-05-08T23:33:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.509+0000 7fe9922ae700 1 -- start start 2024-05-08T23:33:36.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe9922ae700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe98c15b360 0x7fe98c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe9922ae700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 0x7fe98c0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe9922ae700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c0711c0 con 0x7fe98c0c9220 2024-05-08T23:33:36.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe9922ae700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c06e320 con 0x7fe98c15b360 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe9922ae700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0730d0 con 0x7fe98c0b1ad0 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe98c15b360 0x7fe98c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe99084b700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 msgr2=0x7fe98c0b1f40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe98c0c9220 0x7fe98c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52010/0 (socket says 172.21.15.33:52010) 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98b7fe700 1 -- 172.21.15.33:0/1150764525 learned_addr learned my addr 172.21.15.33:0/1150764525 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe99084b700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 0x7fe98c0b1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98bfff700 1 -- 172.21.15.33:0/1150764525 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 msgr2=0x7fe98c0b1f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98bfff700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 0x7fe98c0b1f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98bfff700 1 -- 172.21.15.33:0/1150764525 >> v1:172.21.15.33:6789/0 conn(0x7fe98c0c9220 legacy=0x7fe98c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.510+0000 7fe98bfff700 1 -- 172.21.15.33:0/1150764525 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c071710 con 0x7fe98c15b360 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.511+0000 7fe98bfff700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c15b360 0x7fe98c06a990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009840 tx=0x7fe97c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e7c6971d9f51b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.511+0000 7fe98affd700 1 -- 172.21.15.33:0/1150764525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011830 con 0x7fe98c15b360 2024-05-08T23:33:36.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.511+0000 7fe98affd700 1 -- 172.21.15.33:0/1150764525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c011990 con 0x7fe98c15b360 2024-05-08T23:33:36.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.511+0000 7fe98affd700 1 -- 172.21.15.33:0/1150764525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011d20 con 0x7fe98c15b360 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 -- 172.21.15.33:0/1150764525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c15b360 msgr2=0x7fe98c06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c15b360 0x7fe98c06a990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009840 tx=0x7fe97c009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 -- 172.21.15.33:0/1150764525 shutdown_connections 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c0b1ad0 0x7fe98c0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 --2- 172.21.15.33:0/1150764525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c15b360 0x7fe98c06a990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 -- 172.21.15.33:0/1150764525 >> 172.21.15.33:0/1150764525 conn(0x7fe98c06d2b0 msgr2=0x7fe98c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 -- 172.21.15.33:0/1150764525 shutdown_connections 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.512+0000 7fe9922ae700 1 -- 172.21.15.33:0/1150764525 wait complete. 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.513+0000 7fe9922ae700 1 Processor -- start 2024-05-08T23:33:36.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.513+0000 7fe9922ae700 1 -- start start 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b1ad0 0x7fe98c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 0x7fe98c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0c6b00 con 0x7fe98c0c9220 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c1f39e0 con 0x7fe98c0b1ad0 2024-05-08T23:33:36.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe9922ae700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c1f4840 con 0x7fe98c15b360 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe98b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe98b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41872/0 (socket says 172.21.15.33:41872) 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe98b7fe700 1 -- 172.21.15.33:0/4139562608 learned_addr learned my addr 172.21.15.33:0/4139562608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe99084b700 1 -- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 msgr2=0x7fe98c155cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe99084b700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 0x7fe98c155cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.514+0000 7fe98bfff700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b1ad0 0x7fe98c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 -- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 msgr2=0x7fe98c155cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 0x7fe98c155cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 -- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b1ad0 msgr2=0x7fe98c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b1ad0 0x7fe98c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 -- 172.21.15.33:0/4139562608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c0094a0 con 0x7fe98c0c9220 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe98b7fe700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c550 tx=0x7fe98000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980005d70 con 0x7fe98c0c9220 2024-05-08T23:33:36.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9800070d0 con 0x7fe98c0c9220 2024-05-08T23:33:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c1f3c30 con 0x7fe98c0c9220 2024-05-08T23:33:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.515+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980007440 con 0x7fe98c0c9220 2024-05-08T23:33:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.516+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c1f41f0 con 0x7fe98c0c9220 2024-05-08T23:33:36.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.517+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c155600 con 0x7fe98c0c9220 2024-05-08T23:33:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.517+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe9800075a0 con 0x7fe98c0c9220 2024-05-08T23:33:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.518+0000 7fe988ff9700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 0x7fe97806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.518+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fe980088760 con 0x7fe98c0c9220 2024-05-08T23:33:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.518+0000 7fe98bfff700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 0x7fe97806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.519+0000 7fe98bfff700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 0x7fe97806d100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:36.526+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe980053920 con 0x7fe98c0c9220 2024-05-08T23:33:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.084+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe98c075460 con 0x7fe98c0c9220 2024-05-08T23:33:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.085+0000 7fe988ff9700 1 -- 172.21.15.33:0/4139562608 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fe980056f40 con 0x7fe98c0c9220 2024-05-08T23:33:37.085 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 msgr2=0x7fe97806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 0x7fe97806d100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c01c040 comp rx=0 tx=0).stop 2024-05-08T23:33:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 msgr2=0x7fe98c1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c550 tx=0x7fe98000c860 comp rx=0 tx=0).stop 2024-05-08T23:33:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 shutdown_connections 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe97806ac40 0x7fe97806d100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe98c15b360 0x7fe98c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b1ad0 0x7fe98c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 --2- 172.21.15.33:0/4139562608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe98c0c9220 0x7fe98c1557b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.088+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 >> 172.21.15.33:0/4139562608 conn(0x7fe98c06d2b0 msgr2=0x7fe98c0cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.089+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 shutdown_connections 2024-05-08T23:33:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:37.089+0000 7fe9922ae700 1 -- 172.21.15.33:0/4139562608 wait complete. 2024-05-08T23:33:40.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:33:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.244+0000 7f8554f3c700 1 Processor -- start 2024-05-08T23:33:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.245+0000 7f8554f3c700 1 -- start start 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f8554f3c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f8554f3c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f8554f3c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500c2860 con 0x7f855006d980 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f8554f3c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500c3540 con 0x7f8550154540 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f8554f3c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500c9600 con 0x7f85500712f0 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41888/0 (socket says 172.21.15.33:41888) 2024-05-08T23:33:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 learned_addr learned my addr 172.21.15.33:0/2686017019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.246+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:40.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:40.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.447+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:40.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:40.848+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:33:41.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:41.649+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:33:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 msgr2=0x7f855015d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.42:3300/0 conn(0x7f85500712f0 0x7f855015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 -- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 msgr2=0x7f85500c90c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 --2- 172.21.15.33:0/2686017019 >> v2:172.21.15.33:3300/0 conn(0x7f8550154540 0x7f85500c90c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 -- 172.21.15.33:0/2686017019 >> v1:172.21.15.42:6789/0 conn(0x7f855006d980 legacy=0x7f855015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8530009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 -- 172.21.15.33:0/2686017019 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85500c8620 con 0x7f8530003700 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f853ffff700 1 -- 172.21.15.33:0/2686017019 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500c3540 con 0x7f8530007350 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8530009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.246+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8530009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.447+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8530009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.447+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8530009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:33:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.848+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8530009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.848+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8530009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:33:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.893+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925831722 0 0) 0x7f85500c8620 con 0x7f8530003700 2024-05-08T23:33:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.894+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85380035b0 con 0x7f8530003700 2024-05-08T23:33:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255289572 0 0) 0x7f85380035b0 con 0x7f8530003700 2024-05-08T23:33:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8530009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8530009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853000b030 con 0x7f8530003700 2024-05-08T23:33:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f854000c570 con 0x7f8530003700 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 >> v1:172.21.15.33:6789/0 conn(0x7f8530003700 legacy=0x7f8530005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f8538003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8538004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8538004de0 0x7f8538005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500c3540 con 0x7f8538003a00 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853800bc90 con 0x7f8538004de0 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853800be50 con 0x7f8530007350 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854ed9d700 1 -- 172.21.15.33:0/2686017019 reap_dead start 2024-05-08T23:33:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854dd9b700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8538004850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854dd9b700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8538004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f8538003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.895+0000 7f854ed9d700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8538004de0 0x7f8538005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 msgr2=0x7f8538004850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8538004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8538004de0 msgr2=0x7f8538005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8538004de0 0x7f8538005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 -- 172.21.15.33:0/2686017019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380035b0 con 0x7f8538003a00 2024-05-08T23:33:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.896+0000 7f854e59c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f8538003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8540009180 tx=0x7f8538005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b2f45382c3b059 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.898+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540003b60 con 0x7f8538003a00 2024-05-08T23:33:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.898+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854000c040 con 0x7f8538003a00 2024-05-08T23:33:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.898+0000 7f854d59a700 1 -- 172.21.15.33:0/2686017019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400044d0 con 0x7f8538003a00 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 -- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 msgr2=0x7f8538003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f8538003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8540009180 tx=0x7f8538005880 comp rx=0 tx=0).stop 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 -- 172.21.15.33:0/2686017019 shutdown_connections 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8538004de0 0x7f8538005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8530007350 0x7f8538004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 --2- 172.21.15.33:0/2686017019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f8538003e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.899+0000 7f8554f3c700 1 -- 172.21.15.33:0/2686017019 >> 172.21.15.33:0/2686017019 conn(0x7f855006ce60 msgr2=0x7f855006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.900+0000 7f8554f3c700 1 -- 172.21.15.33:0/2686017019 shutdown_connections 2024-05-08T23:33:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.900+0000 7f8554f3c700 1 -- 172.21.15.33:0/2686017019 wait complete. 2024-05-08T23:33:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.902+0000 7f8554f3c700 1 Processor -- start 2024-05-08T23:33:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.902+0000 7f8554f3c700 1 -- start start 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8530007350 0x7f855015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 0x7f855015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500c4680 con 0x7f8538003a00 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550158b50 con 0x7f8530007350 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f8554f3c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500c70f0 con 0x7f8538004de0 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854ed9d700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 msgr2=0x7f855015ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41920/0 (socket says 172.21.15.33:41920) 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854ed9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 0x7f855015ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854dd9b700 1 -- 172.21.15.33:0/3329481784 learned_addr learned my addr 172.21.15.33:0/3329481784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.903+0000 7f854e59c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8530007350 0x7f855015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 -- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 msgr2=0x7f855015ce70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 0x7f855015ce70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 -- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8530007350 msgr2=0x7f855015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8530007350 0x7f855015c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 -- 172.21.15.33:0/3329481784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855015d7e0 con 0x7f8538003a00 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f854dd9b700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8544010670 tx=0x7f8544010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85440168a0 con 0x7f8538003a00 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854400c030 con 0x7f8538003a00 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544016c40 con 0x7f8538003a00 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855015dac0 con 0x7f8538003a00 2024-05-08T23:33:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.904+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85500c42a0 con 0x7f8538003a00 2024-05-08T23:33:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.906+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550073770 con 0x7f8538003a00 2024-05-08T23:33:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.907+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f854402a480 con 0x7f8538003a00 2024-05-08T23:33:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.907+0000 7f853effd700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 0x7f853006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.907+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f854408a1e0 con 0x7f8538003a00 2024-05-08T23:33:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.907+0000 7f854e59c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 0x7f853006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.908+0000 7f854e59c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 0x7f853006dcf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8540001eb0 tx=0x7f8540017000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:43.915+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8544055240 con 0x7f8538003a00 2024-05-08T23:33:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.322+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f85500c47c0 con 0x7f8538003a00 2024-05-08T23:33:44.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.323+0000 7f853effd700 1 -- 172.21.15.33:0/3329481784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f8544025610 con 0x7f8538003a00 2024-05-08T23:33:44.323 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":20,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 msgr2=0x7f853006dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 0x7f853006dcf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8540001eb0 tx=0x7f8540017000 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 msgr2=0x7f85500c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8544010670 tx=0x7f8544010980 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 shutdown_connections 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f853006b830 0x7f853006dcf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8538004de0 0x7f855015ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8530007350 0x7f855015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 --2- 172.21.15.33:0/3329481784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538003a00 0x7f85500c6580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 >> 172.21.15.33:0/3329481784 conn(0x7f855006ce60 msgr2=0x7f855006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:44.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 shutdown_connections 2024-05-08T23:33:44.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.326+0000 7f8554f3c700 1 -- 172.21.15.33:0/3329481784 wait complete. 2024-05-08T23:33:44.340 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:33:44.340 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-08T23:33:44.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:33:44.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.481+0000 7ff14d8ae700 1 Processor -- start 2024-05-08T23:33:44.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 -- start start 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff148154480 0x7ff1480cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1480c77d0 0x7ff1480c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480cc270 con 0x7ff1480cb980 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1480ccb90 con 0x7ff148154480 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.482+0000 7ff14d8ae700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14806e560 con 0x7ff1480c77d0 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff148154480 0x7ff1480cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff1467fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1480cb980 0x7ff14815d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52024/0 (socket says 172.21.15.33:52024) 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff1467fc700 1 -- 172.21.15.33:0/4000366655 learned_addr learned my addr 172.21.15.33:0/4000366655 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff1477fe700 1 --2- 172.21.15.33:0/4000366655 >> v2:172.21.15.42:3300/0 conn(0x7ff1480c77d0 0x7ff1480c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 -- 172.21.15.33:0/4000366655 >> v2:172.21.15.42:3300/0 conn(0x7ff1480c77d0 msgr2=0x7ff1480c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 --2- 172.21.15.33:0/4000366655 >> v2:172.21.15.42:3300/0 conn(0x7ff1480c77d0 0x7ff1480c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 -- 172.21.15.33:0/4000366655 >> v1:172.21.15.33:6789/0 conn(0x7ff1480cb980 legacy=0x7ff14815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff145ffb700 1 -- 172.21.15.33:0/4000366655 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343501476 0 0) 0x7ff1480cc270 con 0x7ff1480cb980 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 -- 172.21.15.33:0/4000366655 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480cc420 con 0x7ff148154480 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.483+0000 7ff146ffd700 1 --2- 172.21.15.33:0/4000366655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff1480cb200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff14815e910 tx=0x7ff138009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be97d6e8cb038e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.484+0000 7ff145ffb700 1 -- 172.21.15.33:0/4000366655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138011830 con 0x7ff148154480 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.484+0000 7ff145ffb700 1 -- 172.21.15.33:0/4000366655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff138011990 con 0x7ff148154480 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.484+0000 7ff145ffb700 1 -- 172.21.15.33:0/4000366655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138011d20 con 0x7ff148154480 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.484+0000 7ff14d8ae700 1 -- 172.21.15.33:0/4000366655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 msgr2=0x7ff1480cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.484+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/4000366655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff1480cb200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff14815e910 tx=0x7ff138009b20 comp rx=0 tx=0).stop 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 -- 172.21.15.33:0/4000366655 shutdown_connections 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/4000366655 >> v2:172.21.15.42:3300/0 conn(0x7ff1480c77d0 0x7ff1480c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/4000366655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff1480cb200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 -- 172.21.15.33:0/4000366655 >> 172.21.15.33:0/4000366655 conn(0x7ff14806eb10 msgr2=0x7ff14806ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 -- 172.21.15.33:0/4000366655 shutdown_connections 2024-05-08T23:33:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.485+0000 7ff14d8ae700 1 -- 172.21.15.33:0/4000366655 wait complete. 2024-05-08T23:33:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.486+0000 7ff14d8ae700 1 Processor -- start 2024-05-08T23:33:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.486+0000 7ff14d8ae700 1 -- start start 2024-05-08T23:33:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1480c77d0 0x7ff148157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1480cb980 0x7ff1480c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff148149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14815a7f0 con 0x7ff148154480 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1480c4380 con 0x7ff1480c77d0 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff14d8ae700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1480c4f40 con 0x7ff1480cb980 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff1477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff148149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff146ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1480c77d0 0x7ff148157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff1477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff148149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41932/0 (socket says 172.21.15.33:41932) 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.487+0000 7ff1477fe700 1 -- 172.21.15.33:0/1965940745 learned_addr learned my addr 172.21.15.33:0/1965940745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1480cb980 msgr2=0x7ff1480c35b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff1467fc700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1480cb980 msgr2=0x7ff1480c35b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff1467fc700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1480cb980 0x7ff1480c35b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1480cb980 0x7ff1480c35b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 msgr2=0x7ff148149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff148149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1380094a0 con 0x7ff1480c77d0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff1477fe700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148154480 0x7ff148149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff146ffd700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1480c77d0 0x7ff148157460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff148070c00 tx=0x7ff138004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1380036a0 con 0x7ff1480c77d0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13801f070 con 0x7ff1480c77d0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.488+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138003a40 con 0x7ff1480c77d0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.489+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff148159b70 con 0x7ff1480c77d0 2024-05-08T23:33:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.489+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14815a130 con 0x7ff1480c77d0 2024-05-08T23:33:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.490+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 0x7ff14814a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.490+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148074b30 con 0x7ff14815a310 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.490+0000 7ff1467fc700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 0x7ff14814a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.490+0000 7ff1467fc700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff14806f7d0 con 0x7ff14815a310 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff1467fc700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 0x7ff14814a380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff148157dd0 tx=0x7ff13c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c004d40 con 0x7ff14815a310 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff13c0070d0 con 0x7ff14815a310 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 msgr2=0x7ff14814a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 0x7ff14814a380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff148157dd0 tx=0x7ff13c00ca10 comp rx=0 tx=0).stop 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 msgr2=0x7ff14814a380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff14815a310 0x7ff14814a380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.491+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff138003ba0 con 0x7ff1480c77d0 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.492+0000 7ff12ffff700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13406a940 0x7ff13406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.492+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff138021070 con 0x7ff1480c77d0 2024-05-08T23:33:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.494+0000 7ff1467fc700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13406a940 0x7ff13406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.495+0000 7ff1467fc700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13406a940 0x7ff13406ce00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1481f84f0 tx=0x7ff13c00a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.506+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 0x7ff1480c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.506+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14806e280 con 0x7ff148156f10 2024-05-08T23:33:44.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.506+0000 7ff1477fe700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 0x7ff1480c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.506+0000 7ff1477fe700 1 -- 172.21.15.33:0/1965940745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff14806fab0 con 0x7ff148156f10 2024-05-08T23:33:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.506+0000 7ff1477fe700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 0x7ff1480c5220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff140007560 tx=0x7ff140007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140007ed0 con 0x7ff148156f10 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4958 (secure 0 0 0) 0x7ff14000f040 con 0x7ff148156f10 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 msgr2=0x7ff1480c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 0x7ff1480c5220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff140007560 tx=0x7ff140007920 comp rx=0 tx=0).stop 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 msgr2=0x7ff1480c5220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff12ffff700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff148156f10 0x7ff1480c5220 secure :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff140007560 tx=0x7ff140007920 comp rx=0 tx=0).stop 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:33:44.507 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 20, 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.509 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.511 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.512 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.513 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.515 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.516 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.517 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.518 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:44.520 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13406a940 msgr2=0x7ff13406ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13406a940 0x7ff13406ce00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1481f84f0 tx=0x7ff13c00a000 comp rx=0 tx=0).stop 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1480c77d0 msgr2=0x7ff148157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 --2- 172.21.15.33:0/1965940745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1480c77d0 0x7ff148157460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff148070c00 tx=0x7ff138004750 comp rx=0 tx=0).stop 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff1477fe700 1 -- 172.21.15.33:0/1965940745 reap_dead start 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 shutdown_connections 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.507+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 >> 172.21.15.33:0/1965940745 conn(0x7ff14806eb10 msgr2=0x7ff1480cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.509+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 shutdown_connections 2024-05-08T23:33:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.512+0000 7ff14d8ae700 1 -- 172.21.15.33:0/1965940745 wait complete. 2024-05-08T23:33:44.525 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:33:44.525 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:33:44.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:33:44.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.665+0000 7f4090105700 1 Processor -- start 2024-05-08T23:33:44.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 -- start start 2024-05-08T23:33:44.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f40880c2180 0x7f40880c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 0x7f4088070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4088070010 con 0x7f408806b660 2024-05-08T23:33:44.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40880c2aa0 con 0x7f40880c2180 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.666+0000 7f4090105700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4088070870 con 0x7f40880c8e50 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408d6a0700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 msgr2=0x7f4088070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408e6a2700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f408806b660 0x7f4088071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52040/0 (socket says 172.21.15.33:52040) 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408e6a2700 1 -- 172.21.15.33:0/4195573542 learned_addr learned my addr 172.21.15.33:0/4195573542 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408d6a0700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 0x7f4088070ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408dea1700 1 --2- 172.21.15.33:0/4195573542 >> v2:172.21.15.42:3300/0 conn(0x7f40880c2180 0x7f40880c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408ce9f700 1 -- 172.21.15.33:0/4195573542 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841173577 0 0) 0x7f4088070010 con 0x7f408806b660 2024-05-08T23:33:44.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408ce9f700 1 -- 172.21.15.33:0/4195573542 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40740057c0 con 0x7f408806b660 2024-05-08T23:33:44.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408dea1700 1 -- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 msgr2=0x7f4088070ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408dea1700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 0x7f4088070ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408dea1700 1 -- 172.21.15.33:0/4195573542 >> v1:172.21.15.33:6789/0 conn(0x7f408806b660 legacy=0x7f4088071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.667+0000 7f408dea1700 1 -- 172.21.15.33:0/4195573542 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088071020 con 0x7f40880c2180 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.668+0000 7f408dea1700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c2180 0x7f40880c2560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4084009840 tx=0x7f4084009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=216f1ead86d3114 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.668+0000 7f408ce9f700 1 -- 172.21.15.33:0/4195573542 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084011830 con 0x7f40880c2180 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.668+0000 7f408ce9f700 1 -- 172.21.15.33:0/4195573542 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084011990 con 0x7f40880c2180 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.668+0000 7f408ce9f700 1 -- 172.21.15.33:0/4195573542 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084011d20 con 0x7f40880c2180 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 -- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c2180 msgr2=0x7f40880c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c2180 0x7f40880c2560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4084009840 tx=0x7f4084009b50 comp rx=0 tx=0).stop 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 -- 172.21.15.33:0/4195573542 shutdown_connections 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c2180 0x7f40880c2560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 --2- 172.21.15.33:0/4195573542 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c8e50 0x7f4088070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 -- 172.21.15.33:0/4195573542 >> 172.21.15.33:0/4195573542 conn(0x7f40880c8360 msgr2=0x7f40880c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 -- 172.21.15.33:0/4195573542 shutdown_connections 2024-05-08T23:33:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.669+0000 7f4090105700 1 -- 172.21.15.33:0/4195573542 wait complete. 2024-05-08T23:33:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.670+0000 7f4090105700 1 Processor -- start 2024-05-08T23:33:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.670+0000 7f4090105700 1 -- start start 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 0x7f4088153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c2180 0x7f40881efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c8e50 0x7f40881f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881526c0 con 0x7f408806b660 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4088159c50 con 0x7f40880c8e50 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f4090105700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40880c1dc0 con 0x7f40880c2180 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f408dea1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 0x7f4088153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f408d6a0700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c2180 msgr2=0x7f40881efd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f408d6a0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c2180 0x7f40881efd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f408dea1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 0x7f4088153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41952/0 (socket says 172.21.15.33:41952) 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.671+0000 7f408dea1700 1 -- 172.21.15.33:0/1994159569 learned_addr learned my addr 172.21.15.33:0/1994159569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408e6a2700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c8e50 0x7f40881f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c2180 msgr2=0x7f40881efd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f40880c2180 0x7f40881efd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c8e50 msgr2=0x7f40881f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c8e50 0x7f40881f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840094a0 con 0x7f408806b660 2024-05-08T23:33:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408e6a2700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40880c8e50 0x7f40881f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f408dea1700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 0x7f4088153a90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4084009b90 tx=0x7f4084004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840038a0 con 0x7f408806b660 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408401e070 con 0x7f408806b660 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4088156f50 con 0x7f408806b660 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40880c1a50 con 0x7f408806b660 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.672+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084003c00 con 0x7f408806b660 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.673+0000 7f4090105700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 0x7f4070004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.673+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4070007f20 con 0x7f4070002a40 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.674+0000 7f408d6a0700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 0x7f4070004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.674+0000 7f408d6a0700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40700080e0 con 0x7f4070002a40 2024-05-08T23:33:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.674+0000 7f408d6a0700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 0x7f4070004f00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40880c76b0 tx=0x7f4078009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.674+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078003a90 con 0x7f4070002a40 2024-05-08T23:33:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.675+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f408402d450 con 0x7f408806b660 2024-05-08T23:33:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.675+0000 7f407e7fc700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f406c06ad30 0x7f406c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4078003bf0 con 0x7f4070002a40 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 msgr2=0x7f4070004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 0x7f4070004f00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40880c76b0 tx=0x7f4078009f30 comp rx=0 tx=0).stop 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 msgr2=0x7f4070004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070002a40 0x7f4070004f00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.676+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f408408a400 con 0x7f408806b660 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.677+0000 7f408e6a2700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f406c06ad30 0x7f406c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.677+0000 7f408e6a2700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f406c06ad30 0x7f406c06d1f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40800012e0 tx=0x7f4080009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.684+0000 7f4090105700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 0x7f4070001a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:33:44.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.684+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407000a480 con 0x7f4070001580 2024-05-08T23:33:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.684+0000 7f408d6a0700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 0x7f4070001a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:33:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f408d6a0700 1 -- 172.21.15.33:0/1994159569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4070001f50 con 0x7f4070001580 2024-05-08T23:33:44.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f408d6a0700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 0x7f4070001a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40881ed640 tx=0x7f4078005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407801a990 con 0x7f4070001580 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4759 (secure 0 0 0) 0x7f407801aaf0 con 0x7f4070001580 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 msgr2=0x7f4070001a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 0x7f4070001a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40881ed640 tx=0x7f4078005ab0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 msgr2=0x7f4070001a10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.685+0000 7f407e7fc700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4070001580 0x7f4070001a10 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40881ed640 tx=0x7f4078005ab0 comp rx=0 tx=0).stop 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:33:44.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 20, 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.686 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:33:44.688 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:33:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:33:44.690 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.691 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:33:44.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.696 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:33:44.697 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f406c06ad30 msgr2=0x7f406c06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f406c06ad30 0x7f406c06d1f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f40800012e0 tx=0x7f4080009040 comp rx=0 tx=0).stop 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 msgr2=0x7f4088153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 --2- 172.21.15.33:0/1994159569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408806b660 0x7f4088153a90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4084009b90 tx=0x7f4084004750 comp rx=0 tx=0).stop 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f408e6a2700 1 -- 172.21.15.33:0/1994159569 reap_dead start 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f408e6a2700 1 -- 172.21.15.33:0/1994159569 reap_dead start 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 shutdown_connections 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 >> 172.21.15.33:0/1994159569 conn(0x7f40880c8360 msgr2=0x7f40880c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 shutdown_connections 2024-05-08T23:33:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:33:44.686+0000 7f4090105700 1 -- 172.21.15.33:0/1994159569 wait complete. 2024-05-08T23:33:44.699 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:33:44.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:33:47.292 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:33:47.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:02.594 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 85% (10531/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T23:34:03.465 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-05-08T23:34:03.466 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.467 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T23:34:03.467 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T23:34:03.467 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:34:03.472 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T23:34:03.501 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T23:34:03.503 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:34:03.503 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T23:34:04.002 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T23:34:04.072 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T23:34:04.115 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:34:04.117 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:34:04.117 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T23:34:04.158 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:34:04.162 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:34:04.162 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T23:34:04.222 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-05-08T23:34:04.223 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-05-08T23:34:04.224 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-05-08T23:34:04.297 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into 'ffsb'... 2024-05-08T23:34:04.700 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T23:34:04.700 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-08T23:34:04.701 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T23:34:04.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T23:34:04.703 INFO:tasks.ceph.mon.c:Started 2024-05-08T23:34:04.704 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:34:04.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.850+0000 7f3cad893700 1 Processor -- start 2024-05-08T23:34:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.851+0000 7f3cad893700 1 -- start start 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.851+0000 7f3cad893700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3ca80c8e50 0x7f3ca80c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.851+0000 7f3cad893700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca8070010 con 0x7f3ca806b660 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.851+0000 7f3cad893700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca8070870 con 0x7f3ca80c2180 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.851+0000 7f3cad893700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca80c2aa0 con 0x7f3ca80c8e50 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3ca806b660 0x7f3ca8071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54956/0 (socket says 172.21.15.33:54956) 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca77fe700 1 -- 172.21.15.33:0/480636862 learned_addr learned my addr 172.21.15.33:0/480636862 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca67fc700 1 --2- 172.21.15.33:0/480636862 >> v2:172.21.15.42:3300/0 conn(0x7f3ca80c8e50 0x7f3ca80c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332168976 0 0) 0x7f3ca8070010 con 0x7f3ca806b660 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c940034d0 con 0x7f3ca806b660 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450156123 0 0) 0x7f3ca8070870 con 0x7f3ca80c2180 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca8070010 con 0x7f3ca80c2180 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca67fc700 1 -- 172.21.15.33:0/480636862 >> v1:172.21.15.42:6789/0 conn(0x7f3ca80c2180 legacy=0x7f3ca8070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca67fc700 1 -- 172.21.15.33:0/480636862 >> v1:172.21.15.33:6789/0 conn(0x7f3ca806b660 legacy=0x7f3ca8071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca67fc700 1 -- 172.21.15.33:0/480636862 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca815a820 con 0x7f3ca80c8e50 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.852+0000 7f3ca67fc700 1 --2- 172.21.15.33:0/480636862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca80c8e50 0x7f3ca80c2560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009840 tx=0x7f3c98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c0e2c5c5d05f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98011830 con 0x7f3ca80c8e50 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98011990 con 0x7f3ca80c8e50 2024-05-08T23:34:04.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3ca5ffb700 1 -- 172.21.15.33:0/480636862 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98011d20 con 0x7f3ca80c8e50 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 -- 172.21.15.33:0/480636862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca80c8e50 msgr2=0x7f3ca80c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 --2- 172.21.15.33:0/480636862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca80c8e50 0x7f3ca80c2560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009840 tx=0x7f3c98009b50 comp rx=0 tx=0).stop 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 -- 172.21.15.33:0/480636862 shutdown_connections 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 --2- 172.21.15.33:0/480636862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca80c8e50 0x7f3ca80c2560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 -- 172.21.15.33:0/480636862 >> 172.21.15.33:0/480636862 conn(0x7f3ca80c8360 msgr2=0x7f3ca80c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 -- 172.21.15.33:0/480636862 shutdown_connections 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.853+0000 7f3cad893700 1 -- 172.21.15.33:0/480636862 wait complete. 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.855+0000 7f3cad893700 1 Processor -- start 2024-05-08T23:34:04.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.855+0000 7f3cad893700 1 -- start start 2024-05-08T23:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca806b660 0x7f3ca814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:04.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8155580 con 0x7f3ca80c8e50 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8157080 con 0x7f3ca806b660 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3cad893700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8154920 con 0x7f3ca80c2180 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59612/0 (socket says 172.21.15.33:59612) 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 -- 172.21.15.33:0/364796968 learned_addr learned my addr 172.21.15.33:0/364796968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca67fc700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca6ffd700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca806b660 0x7f3ca814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca67fc700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 msgr2=0x7f3ca80c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca67fc700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 msgr2=0x7f3ca80c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca67fc700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca67fc700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 msgr2=0x7f3ca80c6760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca806b660 msgr2=0x7f3ca814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca806b660 0x7f3ca814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:04.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 -- 172.21.15.33:0/364796968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980094a0 con 0x7f3ca80c8e50 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3ca77fe700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000caa0 tx=0x7f3ca000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000a9e0 con 0x7f3ca80c8e50 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.856+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca000ab40 con 0x7f3ca80c8e50 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.857+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0012400 con 0x7f3ca80c8e50 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.857+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca80c9910 con 0x7f3ca80c8e50 2024-05-08T23:34:04.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.857+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca8154be0 con 0x7f3ca80c8e50 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.857+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca8075460 con 0x7f3ca80c8e50 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.858+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3ca001b3b0 con 0x7f3ca80c8e50 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.858+0000 7f3c9ffff700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 0x7f3c8806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.859+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3ca0089020 con 0x7f3ca80c8e50 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.859+0000 7f3ca6ffd700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 0x7f3c8806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:04.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.859+0000 7f3ca6ffd700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 0x7f3c8806d180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c900011e0 tx=0x7f3c90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:04.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:04.863+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3ca00541e0 con 0x7f3ca80c8e50 2024-05-08T23:34:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.190+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3ca80c71f0 con 0x7f3ca80c8e50 2024-05-08T23:34:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.191+0000 7f3c9ffff700 1 -- 172.21.15.33:0/364796968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f3ca0057800 con 0x7f3ca80c8e50 2024-05-08T23:34:05.189 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 msgr2=0x7f3c8806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 0x7f3c8806d180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3c900011e0 tx=0x7f3c90009040 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 msgr2=0x7f3ca81f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000caa0 tx=0x7f3ca000cdb0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 shutdown_connections 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c8806acc0 0x7f3c8806d180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ca80c8e50 0x7f3ca81f31b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ca806b660 0x7f3ca814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 --2- 172.21.15.33:0/364796968 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3ca80c2180 0x7f3ca80c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 >> 172.21.15.33:0/364796968 conn(0x7f3ca80c8360 msgr2=0x7f3ca8150510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 shutdown_connections 2024-05-08T23:34:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.193+0000 7f3cad893700 1 -- 172.21.15.33:0/364796968 wait complete. 2024-05-08T23:34:05.205 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:34:05.205 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:34:05.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:34:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.344+0000 7fe686f0c700 1 Processor -- start 2024-05-08T23:34:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.344+0000 7fe686f0c700 1 -- start start 2024-05-08T23:34:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe686f0c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe68015ad80 0x7fe68006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe686f0c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe686f0c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68015d870 con 0x7fe680154460 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe686f0c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe680071150 con 0x7fe68015ad80 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe686f0c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800c7e30 con 0x7fe68006f810 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe684ca8700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe680154460 0x7fe68015dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54966/0 (socket says 172.21.15.33:54966) 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe684ca8700 1 -- 172.21.15.33:0/801276686 learned_addr learned my addr 172.21.15.33:0/801276686 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe67ffff700 1 --2- 172.21.15.33:0/801276686 >> v2:172.21.15.42:3300/0 conn(0x7fe68015ad80 0x7fe68006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe6854a9700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe6854a9700 1 -- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 msgr2=0x7fe68006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.345+0000 7fe67f7fe700 1 -- 172.21.15.33:0/801276686 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717075176 0 0) 0x7fe68015d870 con 0x7fe680154460 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe6854a9700 1 -- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 msgr2=0x7fe68006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe6854a9700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67f7fe700 1 -- 172.21.15.33:0/801276686 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6700035b0 con 0x7fe680154460 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe6854a9700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67ffff700 1 -- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 msgr2=0x7fe68006aed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67ffff700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67ffff700 1 -- 172.21.15.33:0/801276686 >> v1:172.21.15.33:6789/0 conn(0x7fe680154460 legacy=0x7fe68015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67ffff700 1 -- 172.21.15.33:0/801276686 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68015b220 con 0x7fe68015ad80 2024-05-08T23:34:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67ffff700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68015ad80 0x7fe68006a990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009840 tx=0x7fe66c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4161d4d2f6308602 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67f7fe700 1 -- 172.21.15.33:0/801276686 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c011830 con 0x7fe68015ad80 2024-05-08T23:34:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67f7fe700 1 -- 172.21.15.33:0/801276686 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c011990 con 0x7fe68015ad80 2024-05-08T23:34:05.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.346+0000 7fe67f7fe700 1 -- 172.21.15.33:0/801276686 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c011d00 con 0x7fe68015ad80 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 -- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68015ad80 msgr2=0x7fe68006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68015ad80 0x7fe68006a990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009840 tx=0x7fe66c009b50 comp rx=0 tx=0).stop 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 -- 172.21.15.33:0/801276686 shutdown_connections 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe68006f810 0x7fe68006aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 --2- 172.21.15.33:0/801276686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68015ad80 0x7fe68006a990 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 -- 172.21.15.33:0/801276686 >> 172.21.15.33:0/801276686 conn(0x7fe68006e990 msgr2=0x7fe68006ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 -- 172.21.15.33:0/801276686 shutdown_connections 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.347+0000 7fe686f0c700 1 -- 172.21.15.33:0/801276686 wait complete. 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.348+0000 7fe686f0c700 1 Processor -- start 2024-05-08T23:34:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.348+0000 7fe686f0c700 1 -- start start 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68006f810 0x7fe6800c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe680154460 0x7fe6801e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 0x7fe6801f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801e42a0 con 0x7fe68015ad80 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801e3350 con 0x7fe68006f810 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe686f0c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6801f0610 con 0x7fe680154460 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe6854a9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 0x7fe6801f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe6854a9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 0x7fe6801f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59620/0 (socket says 172.21.15.33:59620) 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.349+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 learned_addr learned my addr 172.21.15.33:0/1534101892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe67ffff700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe680154460 0x7fe6801e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe684ca8700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68006f810 0x7fe6800c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe680154460 msgr2=0x7fe6801e4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe680154460 0x7fe6801e4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68006f810 msgr2=0x7fe6800c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe68006f810 0x7fe6800c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0094a0 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe6854a9700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 0x7fe6801f00d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe678007e30 tx=0x7fe67800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800ed80 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678014420 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68006ad40 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6780147d0 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.350+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6801e3880 con 0x7fe68015ad80 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.351+0000 7fe686f0c700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 0x7fe6801538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.351+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800749b0 con 0x7fe6801e3c50 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.351+0000 7fe684ca8700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 0x7fe6801538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe684ca8700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe680071990 con 0x7fe6801e3c50 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe684ca8700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 0x7fe6801538c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe674000f60 tx=0x7fe67400ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe678014930 con 0x7fe68015ad80 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe67d7fa700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe66406ac90 0x7fe66406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fe678089b50 con 0x7fe68015ad80 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe67ffff700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe66406ac90 0x7fe66406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.352+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674005b50 con 0x7fe6801e3c50 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe674005cb0 con 0x7fe6801e3c50 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 msgr2=0x7fe6801538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67d7fa700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 0x7fe6801538c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe674000f60 tx=0x7fe67400ea90 comp rx=0 tx=0).stop 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 msgr2=0x7fe6801538c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67d7fa700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e3c50 0x7fe6801538c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.353+0000 7fe67ffff700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe66406ac90 0x7fe66406d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009fd0 tx=0x7fe66c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.360+0000 7fe686f0c700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 0x7fe6801e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.360+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800716b0 con 0x7fe6801e43e0 2024-05-08T23:34:05.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe684ca8700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 0x7fe6801e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe684ca8700 1 -- 172.21.15.33:0/1534101892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe68015dac0 con 0x7fe6801e43e0 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe684ca8700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 0x7fe6801e47f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe674000f60 tx=0x7fe6740010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67401cae0 con 0x7fe6801e43e0 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4969 (secure 0 0 0) 0x7fe67401cc40 con 0x7fe6801e43e0 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 msgr2=0x7fe6801e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 0x7fe6801e47f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe674000f60 tx=0x7fe6740010a0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 msgr2=0x7fe6801e47f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.361+0000 7fe67d7fa700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6801e43e0 0x7fe6801e47f0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe674000f60 tx=0x7fe6740010a0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.363 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.365 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:05.367 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.368 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:05.369 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:05.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:05.372 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe66406ac90 msgr2=0x7fe66406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe66406ac90 0x7fe66406d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009fd0 tx=0x7fe66c008030 comp rx=0 tx=0).stop 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 msgr2=0x7fe6801f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 --2- 172.21.15.33:0/1534101892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe68015ad80 0x7fe6801f00d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe678007e30 tx=0x7fe67800e3c0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 reap_dead start 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe6854a9700 1 -- 172.21.15.33:0/1534101892 reap_dead start 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 shutdown_connections 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 >> 172.21.15.33:0/1534101892 conn(0x7fe68006e990 msgr2=0x7fe68006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 shutdown_connections 2024-05-08T23:34:05.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.362+0000 7fe686f0c700 1 -- 172.21.15.33:0/1534101892 wait complete. 2024-05-08T23:34:05.375 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:05.375 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:34:05.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:34:05.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.514+0000 7f43c3f24700 1 Processor -- start 2024-05-08T23:34:05.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.514+0000 7f43c3f24700 1 -- start start 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c3f24700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f43bc1521d0 0x7f43bc158a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c3f24700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43bc0ce330 0x7f43bc0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c3f24700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc06e060 con 0x7f43bc1594b0 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c3f24700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc157fc0 con 0x7f43bc0ce330 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c3f24700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc158580 con 0x7f43bc1521d0 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c24c1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f43bc0ce330 0x7f43bc0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c14bf700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43bc1594b0 0x7f43bc06eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:52778/0 (socket says 172.21.15.33:52778) 2024-05-08T23:34:05.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c14bf700 1 -- 172.21.15.33:0/917536296 learned_addr learned my addr 172.21.15.33:0/917536296 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/917536296 >> v2:172.21.15.42:3300/0 conn(0x7f43bc1521d0 0x7f43bc158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c24c1700 1 -- 172.21.15.33:0/917536296 >> v2:172.21.15.42:3300/0 conn(0x7f43bc1521d0 msgr2=0x7f43bc158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c24c1700 1 --2- 172.21.15.33:0/917536296 >> v2:172.21.15.42:3300/0 conn(0x7f43bc1521d0 0x7f43bc158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c24c1700 1 -- 172.21.15.33:0/917536296 >> v1:172.21.15.33:6789/0 conn(0x7f43bc1594b0 legacy=0x7f43bc06eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c24c1700 1 -- 172.21.15.33:0/917536296 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc158f50 con 0x7f43bc0ce330 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.515+0000 7f43c0cbe700 1 -- 172.21.15.33:0/917536296 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855043375 0 0) 0x7f43bc06e060 con 0x7f43bc1594b0 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c24c1700 1 --2- 172.21.15.33:0/917536296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc0ce330 0x7f43bc0ce7c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0cb980 tx=0x7f43b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc2d491591b68f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c0cbe700 1 -- 172.21.15.33:0/917536296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400e070 con 0x7f43bc0ce330 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c0cbe700 1 -- 172.21.15.33:0/917536296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b400bb20 con 0x7f43bc0ce330 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c0cbe700 1 -- 172.21.15.33:0/917536296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4012430 con 0x7f43bc0ce330 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 -- 172.21.15.33:0/917536296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc0ce330 msgr2=0x7f43bc0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 --2- 172.21.15.33:0/917536296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc0ce330 0x7f43bc0ce7c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0cb980 tx=0x7f43b400b330 comp rx=0 tx=0).stop 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 -- 172.21.15.33:0/917536296 shutdown_connections 2024-05-08T23:34:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 --2- 172.21.15.33:0/917536296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc0ce330 0x7f43bc0ce7c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 --2- 172.21.15.33:0/917536296 >> v2:172.21.15.42:3300/0 conn(0x7f43bc1521d0 0x7f43bc158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 -- 172.21.15.33:0/917536296 >> 172.21.15.33:0/917536296 conn(0x7f43bc072670 msgr2=0x7f43bc072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 -- 172.21.15.33:0/917536296 shutdown_connections 2024-05-08T23:34:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.516+0000 7f43c3f24700 1 -- 172.21.15.33:0/917536296 wait complete. 2024-05-08T23:34:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.518+0000 7f43c3f24700 1 Processor -- start 2024-05-08T23:34:05.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.518+0000 7f43c3f24700 1 -- start start 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 0x7f43bc1473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 0x7f43bc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc1594b0 0x7f43bc0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1496a0 con 0x7f43bc1521d0 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1515a0 con 0x7f43bc1594b0 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c3f24700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1468e0 con 0x7f43bc0ce330 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 0x7f43bc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 0x7f43bc147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38676/0 (socket says 172.21.15.33:38676) 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 -- 172.21.15.33:0/3392551503 learned_addr learned my addr 172.21.15.33:0/3392551503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c24c1700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc1594b0 0x7f43bc0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 0x7f43bc1473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c1cc0700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 msgr2=0x7f43bc1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c1cc0700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 msgr2=0x7f43bc1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 0x7f43bc1473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 0x7f43bc1473d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:05.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 msgr2=0x7f43bc1473d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f43bc0ce330 0x7f43bc1473d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc1594b0 msgr2=0x7f43bc0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc1594b0 0x7f43bc0cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4009a70 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43c14bf700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 0x7f43bc147f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00c540 tx=0x7f43ac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac004c80 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.519+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac004de0 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.520+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc151a30 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.520+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac007310 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.520+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc150c70 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43c3f24700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 0x7f43bc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc06b420 con 0x7f43bc06d450 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 0x7f43bc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f43ac007470 con 0x7f43bc1521d0 2024-05-08T23:34:05.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43c1cc0700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43bc158210 con 0x7f43bc06d450 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 0x7f43bc0cd540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43b8011800 tx=0x7f43b8011bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.521+0000 7f43b27fc700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43a006ac90 0x7f43a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f43ac088fe0 con 0x7f43bc1521d0 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43c24c1700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43a006ac90 0x7f43a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8004490 con 0x7f43bc06d450 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f43b80036a0 con 0x7f43bc06d450 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 msgr2=0x7f43bc0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 0x7f43bc0cd540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43b8011800 tx=0x7f43b8011bc0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43c24c1700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43a006ac90 0x7f43a006d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43b4014040 tx=0x7f43b4011000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 msgr2=0x7f43bc0cd540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:05.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.522+0000 7f43b27fc700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc06d450 0x7f43bc0cd540 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43b8011800 tx=0x7f43b8011bc0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.530+0000 7f43c3f24700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 0x7f43bc150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.530+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc06e590 con 0x7f43bc147910 2024-05-08T23:34:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.531+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 0x7f43bc150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.531+0000 7f43c1cc0700 1 -- 172.21.15.33:0/3392551503 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43bc0c8870 con 0x7f43bc147910 2024-05-08T23:34:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.531+0000 7f43c1cc0700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 0x7f43bc150e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097b0 tx=0x7f43b8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.531+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8003b20 con 0x7f43bc147910 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4770 (secure 0 0 0) 0x7f43b8003c80 con 0x7f43bc147910 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 msgr2=0x7f43bc150e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43b27fc700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 0x7f43bc150e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097b0 tx=0x7f43b8004750 comp rx=0 tx=0).stop 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43b27fc700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 msgr2=0x7f43bc150e60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43b27fc700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43bc147910 0x7f43bc150e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:05.534 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.536 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:05.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:05.542 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43a006ac90 msgr2=0x7f43a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f43a006ac90 0x7f43a006d150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f43b4014040 tx=0x7f43b4011000 comp rx=0 tx=0).stop 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 msgr2=0x7f43bc147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 --2- 172.21.15.33:0/3392551503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43bc1521d0 0x7f43bc147f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00c540 tx=0x7f43ac00c850 comp rx=0 tx=0).stop 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c24c1700 1 -- 172.21.15.33:0/3392551503 reap_dead start 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c24c1700 1 -- 172.21.15.33:0/3392551503 reap_dead start 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 shutdown_connections 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 >> 172.21.15.33:0/3392551503 conn(0x7f43bc072670 msgr2=0x7f43bc072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 shutdown_connections 2024-05-08T23:34:05.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.532+0000 7f43c3f24700 1 -- 172.21.15.33:0/3392551503 wait complete. 2024-05-08T23:34:05.544 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:05.545 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:34:05.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:34:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.684+0000 7fae632dd700 1 Processor -- start 2024-05-08T23:34:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 -- start start 2024-05-08T23:34:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae5c152580 0x7fae5c0c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 0x7fae5c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c071b30 con 0x7fae5c152580 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0c7a90 con 0x7fae5c158fd0 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.685+0000 7fae632dd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c06e240 con 0x7fae5c06c890 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae5c152580 0x7fae5c0c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fae5c152580 0x7fae5c0c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38686/0 (socket says 172.21.15.33:38686) 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 -- 172.21.15.33:0/510344801 learned_addr learned my addr 172.21.15.33:0/510344801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae6187a700 1 --2- 172.21.15.33:0/510344801 >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 0x7fae5c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae60878700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 -- 172.21.15.33:0/510344801 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 msgr2=0x7fae5c158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 -- 172.21.15.33:0/510344801 >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 msgr2=0x7fae5c0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 --2- 172.21.15.33:0/510344801 >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 0x7fae5c0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 -- 172.21.15.33:0/510344801 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c06b4f0 con 0x7fae5c152580 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae6187a700 1 --2- 172.21.15.33:0/510344801 >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 0x7fae5c0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae61079700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c0c7550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae5c067200 tx=0x7fae58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef2f17143c94de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae53fff700 1 -- 172.21.15.33:0/510344801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800b870 con 0x7fae5c152580 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae53fff700 1 -- 172.21.15.33:0/510344801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800b9d0 con 0x7fae5c152580 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.686+0000 7fae53fff700 1 -- 172.21.15.33:0/510344801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800bd60 con 0x7fae5c152580 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 -- 172.21.15.33:0/510344801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 msgr2=0x7fae5c0c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c0c7550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fae5c067200 tx=0x7fae58009bd0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 -- 172.21.15.33:0/510344801 shutdown_connections 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c0c7550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 --2- 172.21.15.33:0/510344801 >> v2:172.21.15.42:3300/0 conn(0x7fae5c158fd0 0x7fae5c0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 --2- 172.21.15.33:0/510344801 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 -- 172.21.15.33:0/510344801 >> 172.21.15.33:0/510344801 conn(0x7fae5c06d2b0 msgr2=0x7fae5c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 -- 172.21.15.33:0/510344801 shutdown_connections 2024-05-08T23:34:05.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.687+0000 7fae632dd700 1 -- 172.21.15.33:0/510344801 wait complete. 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.688+0000 7fae632dd700 1 Processor -- start 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.688+0000 7fae632dd700 1 -- start start 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5c158fd0 0x7fae5c0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0ca090 con 0x7fae5c152580 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c14bdb0 con 0x7fae5c158fd0 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae632dd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c14c000 con 0x7fae5c06c890 2024-05-08T23:34:05.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae60878700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae60878700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38696/0 (socket says 172.21.15.33:38696) 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae60878700 1 -- 172.21.15.33:0/3126919341 learned_addr learned my addr 172.21.15.33:0/3126919341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.689+0000 7fae6187a700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5c158fd0 0x7fae5c0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 msgr2=0x7fae5c1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 msgr2=0x7fae5c1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c1f4150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 msgr2=0x7fae5c1f4150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c06c890 0x7fae5c1f4150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5c158fd0 msgr2=0x7fae5c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae5c158fd0 0x7fae5c0c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009550 con 0x7fae5c152580 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae60878700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e810 tx=0x7fae4c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c013610 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c013790 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c1f3670 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c013b40 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.690+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c1f3c30 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0749b0 con 0x7fae5c0c7ce0 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.691+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.692+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fae4c013ca0 con 0x7fae5c152580 2024-05-08T23:34:05.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.692+0000 7fae51ffb700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 0x7fae4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:05.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.692+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fae4c088fb0 con 0x7fae5c152580 2024-05-08T23:34:05.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.692+0000 7fae6187a700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 0x7fae4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.693+0000 7fae6187a700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 0x7fae4006d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae5c1593e0 tx=0x7fae54009700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:05.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.892+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:05.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.892+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-08T23:34:05.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.892+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:05.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.892+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:05.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:05.892+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:34:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:06.293+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:06.293+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-08T23:34:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:06.293+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:06.293+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:06.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:06.293+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:34:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:07.094+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:07.094+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-08T23:34:07.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:07.094+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:07.094+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:07.095+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:34:07.738 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-08T23:34:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:08.697+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:08.697+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-08T23:34:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:08.697+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:08.697+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:08.697+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T23:34:08.747 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether build environment is sane... yes 2024-05-08T23:34:08.759 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-08T23:34:08.759 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gawk... gawk 2024-05-08T23:34:08.776 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether make sets $(MAKE)... yes 2024-05-08T23:34:08.816 INFO:tasks.workunit.client.0.smithi033.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-08T23:34:08.816 INFO:tasks.workunit.client.0.smithi033.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-08T23:34:08.828 INFO:tasks.workunit.client.0.smithi033.stdout:checking for style of include used by make... GNU 2024-05-08T23:34:08.832 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... gcc 2024-05-08T23:34:08.896 INFO:tasks.workunit.client.0.smithi033.stdout:checking for C compiler default output file name... a.out 2024-05-08T23:34:08.898 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether the C compiler works... yes 2024-05-08T23:34:08.901 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are cross compiling... no 2024-05-08T23:34:08.942 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of executables... 2024-05-08T23:34:08.979 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of object files... o 2024-05-08T23:34:09.013 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T23:34:09.048 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... yes 2024-05-08T23:34:09.095 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-08T23:34:09.353 INFO:tasks.workunit.client.0.smithi033.stdout:checking dependency style of gcc... gcc3 2024-05-08T23:34:09.498 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.689+0000 7fae50ff9700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae50ff9700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae50ff9700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae50ff9700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 msgr2=0x7fae5c1e4a40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae50ff9700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c0c7ce0 0x7fae5c1e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae50ff9700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae34006110 con 0x7fae340035d0 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 msgr2=0x7fae34005a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 msgr2=0x7fae34005a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.690+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:09.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.891+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:09.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.891+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae34006280 con 0x7fae340035d0 2024-05-08T23:34:09.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.891+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae5c067200 tx=0x7fae58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:09.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58004450 con 0x7fae340035d0 2024-05-08T23:34:09.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fae58021040 con 0x7fae340035d0 2024-05-08T23:34:09.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 msgr2=0x7fae34005a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:09.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae5c067200 tx=0x7fae58004750 comp rx=0 tx=0).stop 2024-05-08T23:34:09.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 msgr2=0x7fae34005a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:09.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.892+0000 7fae51ffb700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae340035d0 0x7fae34005a60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.901+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 0x7fae5c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:09.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.901+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c1f3800 con 0x7fae5c1e4100 2024-05-08T23:34:09.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.902+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 0x7fae5c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.902+0000 7fae61079700 1 -- 172.21.15.33:0/3126919341 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fae5c071d80 con 0x7fae5c1e4100 2024-05-08T23:34:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.902+0000 7fae61079700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 0x7fae5c1f2ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae34006250 tx=0x7fae58011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:09.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58036070 con 0x7fae5c1e4100 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4457 (secure 0 0 0) 0x7fae580045b0 con 0x7fae5c1e4100 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 msgr2=0x7fae5c1f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 0x7fae5c1f2ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae34006250 tx=0x7fae58011000 comp rx=0 tx=0).stop 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 msgr2=0x7fae5c1f2ab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae51ffb700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fae5c1e4100 0x7fae5c1f2ab0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae6187a700 1 -- 172.21.15.33:0/3126919341 reap_dead start 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 4, 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:09.913 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 msgr2=0x7fae4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.903+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 0x7fae4006d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fae5c1593e0 tx=0x7fae54009700 comp rx=0 tx=0).stop 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 msgr2=0x7fae5c1f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e810 tx=0x7fae4c00eb20 comp rx=0 tx=0).stop 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 shutdown_connections 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fae4006ac90 0x7fae4006d150 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 --2- 172.21.15.33:0/3126919341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae5c152580 0x7fae5c1f14d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 >> 172.21.15.33:0/3126919341 conn(0x7fae5c06d2b0 msgr2=0x7fae5c150eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 shutdown_connections 2024-05-08T23:34:09.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:09.904+0000 7fae632dd700 1 -- 172.21.15.33:0/3126919341 wait complete. 2024-05-08T23:34:09.920 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:09.920 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:34:09.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:34:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.063+0000 7fd07d606700 1 Processor -- start 2024-05-08T23:34:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.063+0000 7fd07d606700 1 -- start start 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.063+0000 7fd07d606700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd078154770 0x7fd07815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.063+0000 7fd07d606700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd0780d00b0 0x7fd0780d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd07d606700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780c5c20 con 0x7fd0780c4c70 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd07d606700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780b30b0 con 0x7fd078154770 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd07d606700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd078154b50 con 0x7fd0780d00b0 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd078154770 0x7fd07815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd078154770 0x7fd07815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38710/0 (socket says 172.21.15.33:38710) 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 -- 172.21.15.33:0/2636895093 learned_addr learned my addr 172.21.15.33:0/2636895093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd0777fe700 1 --2- 172.21.15.33:0/2636895093 >> v2:172.21.15.42:3300/0 conn(0x7fd0780d00b0 0x7fd0780d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 -- 172.21.15.33:0/2636895093 >> v2:172.21.15.42:3300/0 conn(0x7fd0780d00b0 msgr2=0x7fd0780d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 --2- 172.21.15.33:0/2636895093 >> v2:172.21.15.42:3300/0 conn(0x7fd0780d00b0 0x7fd0780d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 -- 172.21.15.33:0/2636895093 >> v1:172.21.15.42:6789/0 conn(0x7fd0780c4c70 legacy=0x7fd0780c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 -- 172.21.15.33:0/2636895093 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780caed0 con 0x7fd078154770 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd076ffd700 1 --2- 172.21.15.33:0/2636895093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078154770 0x7fd07815ae50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd07808d8a0 tx=0x7fd068009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99ffb9ee604dabd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd075ffb700 1 -- 172.21.15.33:0/2636895093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800b870 con 0x7fd078154770 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd075ffb700 1 -- 172.21.15.33:0/2636895093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800b9d0 con 0x7fd078154770 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.064+0000 7fd075ffb700 1 -- 172.21.15.33:0/2636895093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800bd60 con 0x7fd078154770 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 -- 172.21.15.33:0/2636895093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078154770 msgr2=0x7fd07815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 --2- 172.21.15.33:0/2636895093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078154770 0x7fd07815ae50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd07808d8a0 tx=0x7fd068009bd0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 -- 172.21.15.33:0/2636895093 shutdown_connections 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 --2- 172.21.15.33:0/2636895093 >> v2:172.21.15.42:3300/0 conn(0x7fd0780d00b0 0x7fd0780d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 --2- 172.21.15.33:0/2636895093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd078154770 0x7fd07815ae50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 -- 172.21.15.33:0/2636895093 >> 172.21.15.33:0/2636895093 conn(0x7fd078066ae0 msgr2=0x7fd078066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 -- 172.21.15.33:0/2636895093 shutdown_connections 2024-05-08T23:34:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.065+0000 7fd07d606700 1 -- 172.21.15.33:0/2636895093 wait complete. 2024-05-08T23:34:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 Processor -- start 2024-05-08T23:34:10.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 -- start start 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0780d00b0 0x7fd0781573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd078154770 0x7fd0780c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781f1aa0 con 0x7fd0780c4c70 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c8a60 con 0x7fd0780d00b0 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.067+0000 7fd07d606700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07814a9d0 con 0x7fd078154770 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38714/0 (socket says 172.21.15.33:38714) 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 -- 172.21.15.33:0/62510165 learned_addr learned my addr 172.21.15.33:0/62510165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd0777fe700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd078154770 0x7fd0780c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd0767fc700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0780d00b0 0x7fd0781573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 -- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd078154770 msgr2=0x7fd0780c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd078154770 0x7fd0780c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 -- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0780d00b0 msgr2=0x7fd0781573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0780d00b0 0x7fd0781573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 -- 172.21.15.33:0/62510165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009550 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd076ffd700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd068004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068004520 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06801e070 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680038a0 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780c8310 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.068+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0781f13e0 con 0x7fd0780c4c70 2024-05-08T23:34:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.069+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd078075460 con 0x7fd0780c4c70 2024-05-08T23:34:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.070+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd068003a00 con 0x7fd0780c4c70 2024-05-08T23:34:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.070+0000 7fd06f7fe700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 0x7fd05806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:10.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.071+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd068020070 con 0x7fd0780c4c70 2024-05-08T23:34:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.074+0000 7fd0767fc700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 0x7fd05806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.074+0000 7fd0767fc700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 0x7fd05806d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0780cb210 tx=0x7fd060009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:10.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.076+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd068055b30 con 0x7fd0780c4c70 2024-05-08T23:34:10.247 INFO:tasks.workunit.client.0.smithi033.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-08T23:34:10.248 INFO:tasks.workunit.client.0.smithi033.stdout:checking for egrep... /usr/bin/grep -E 2024-05-08T23:34:10.256 INFO:tasks.workunit.client.0.smithi033.stdout:checking for AIX... no 2024-05-08T23:34:10.304 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... (cached) gcc 2024-05-08T23:34:10.312 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-08T23:34:10.314 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-08T23:34:10.314 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-08T23:34:10.314 INFO:tasks.workunit.client.0.smithi033.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-08T23:34:10.366 INFO:tasks.workunit.client.0.smithi033.stdout:checking for main in -lm... yes 2024-05-08T23:34:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.397+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd0781584d0 con 0x7fd0780c4c70 2024-05-08T23:34:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.397+0000 7fd06f7fe700 1 -- 172.21.15.33:0/62510165 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fd06802a080 con 0x7fd0780c4c70 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 msgr2=0x7fd05806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 0x7fd05806d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0780cb210 tx=0x7fd060009040 comp rx=0 tx=0).stop 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 msgr2=0x7fd0780c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd068004750 comp rx=0 tx=0).stop 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 shutdown_connections 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ac90 0x7fd05806d150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd078154770 0x7fd0780c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0780c4c70 0x7fd0780c7d30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 --2- 172.21.15.33:0/62510165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0780d00b0 0x7fd0781573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 >> 172.21.15.33:0/62510165 conn(0x7fd078066ae0 msgr2=0x7fd0781587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 shutdown_connections 2024-05-08T23:34:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:10.400+0000 7fd07d606700 1 -- 172.21.15.33:0/62510165 wait complete. 2024-05-08T23:34:10.415 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:34:10.420 INFO:tasks.workunit.client.0.smithi033.stdout:checking for main in -lpthread... yes 2024-05-08T23:34:11.198 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ANSI C header files... yes 2024-05-08T23:34:11.379 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-08T23:34:11.432 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/types.h... yes 2024-05-08T23:34:11.490 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/stat.h... yes 2024-05-08T23:34:11.548 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdlib.h... yes 2024-05-08T23:34:11.607 INFO:tasks.workunit.client.0.smithi033.stdout:checking for string.h... yes 2024-05-08T23:34:11.664 INFO:tasks.workunit.client.0.smithi033.stdout:checking for memory.h... yes 2024-05-08T23:34:11.718 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strings.h... yes 2024-05-08T23:34:11.773 INFO:tasks.workunit.client.0.smithi033.stdout:checking for inttypes.h... yes 2024-05-08T23:34:12.086 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdint.h... yes 2024-05-08T23:34:12.146 INFO:tasks.workunit.client.0.smithi033.stdout:checking for unistd.h... yes 2024-05-08T23:34:12.219 INFO:tasks.workunit.client.0.smithi033.stdout:checking pthread.h usability... yes 2024-05-08T23:34:12.242 INFO:tasks.workunit.client.0.smithi033.stdout:checking pthread.h presence... yes 2024-05-08T23:34:12.242 INFO:tasks.workunit.client.0.smithi033.stdout:checking for pthread.h... yes 2024-05-08T23:34:12.303 INFO:tasks.workunit.client.0.smithi033.stdout:checking fcntl.h usability... yes 2024-05-08T23:34:12.324 INFO:tasks.workunit.client.0.smithi033.stdout:checking fcntl.h presence... yes 2024-05-08T23:34:12.324 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fcntl.h... yes 2024-05-08T23:34:12.649 INFO:tasks.workunit.client.0.smithi033.stdout:checking limits.h usability... yes 2024-05-08T23:34:12.928 INFO:tasks.workunit.client.0.smithi033.stdout:checking limits.h presence... yes 2024-05-08T23:34:12.928 INFO:tasks.workunit.client.0.smithi033.stdout:checking for limits.h... yes 2024-05-08T23:34:12.948 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdint.h... (cached) yes 2024-05-08T23:34:12.998 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h usability... yes 2024-05-08T23:34:13.019 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h presence... yes 2024-05-08T23:34:13.019 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/time.h... yes 2024-05-08T23:34:13.038 INFO:tasks.workunit.client.0.smithi033.stdout:checking for unistd.h... (cached) yes 2024-05-08T23:34:13.088 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/vfs.h usability... yes 2024-05-08T23:34:13.108 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/vfs.h presence... yes 2024-05-08T23:34:13.109 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/vfs.h... yes 2024-05-08T23:34:13.164 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/limits.h usability... no 2024-05-08T23:34:13.185 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/limits.h presence... no 2024-05-08T23:34:13.186 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/limits.h... no 2024-05-08T23:34:13.410 INFO:tasks.workunit.client.0.smithi033.stdout:checking for an ANSI C-conforming const... yes 2024-05-08T23:34:13.626 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-08T23:34:13.628 INFO:tasks.workunit.client.0.smithi033.stdout:checking for function prototypes... yes 2024-05-08T23:34:13.659 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether setvbuf arguments are reversed... no 2024-05-08T23:34:13.740 INFO:tasks.workunit.client.0.smithi033.stdout:checking for system... yes 2024-05-08T23:34:13.812 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gettimeofday... yes 2024-05-08T23:34:13.884 INFO:tasks.workunit.client.0.smithi033.stdout:checking for mkdir... yes 2024-05-08T23:34:13.958 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strerror... yes 2024-05-08T23:34:14.043 INFO:tasks.workunit.client.0.smithi033.stdout:checking for open64... yes 2024-05-08T23:34:14.112 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stat64... yes 2024-05-08T23:34:14.449 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fseeko64... yes 2024-05-08T23:34:15.162 INFO:tasks.workunit.client.0.smithi033.stdout:checking for lrand48_r... yes 2024-05-08T23:34:15.238 INFO:tasks.workunit.client.0.smithi033.stdout:checking for srand48_r... yes 2024-05-08T23:34:15.267 INFO:tasks.workunit.client.0.smithi033.stdout:configure: creating ./config.status 2024-05-08T23:34:15.387 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating Makefile 2024-05-08T23:34:15.404 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating config.h 2024-05-08T23:34:15.424 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: executing depfiles commands 2024-05-08T23:34:15.484 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: executing default commands 2024-05-08T23:34:15.547 INFO:tasks.workunit.client.0.smithi033.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-05-08T23:34:15.550 INFO:tasks.workunit.client.0.smithi033.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-05-08T23:34:15.551 INFO:tasks.workunit.client.0.smithi033.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-05-08T23:34:15.551 INFO:tasks.workunit.client.0.smithi033.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-05-08T23:34:15.551 INFO:tasks.workunit.client.0.smithi033.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-05-08T23:34:15.551 INFO:tasks.workunit.client.0.smithi033.stderr: any GNU archive site. 2024-05-08T23:34:15.553 INFO:tasks.workunit.client.0.smithi033.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-05-08T23:34:15.555 INFO:tasks.workunit.client.0.smithi033.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-05-08T23:34:15.556 INFO:tasks.workunit.client.0.smithi033.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-05-08T23:34:15.556 INFO:tasks.workunit.client.0.smithi033.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-05-08T23:34:15.556 INFO:tasks.workunit.client.0.smithi033.stderr: You might want to install the `Automake' and `Perl' packages. 2024-05-08T23:34:15.556 INFO:tasks.workunit.client.0.smithi033.stderr: Grab them from any GNU archive site. 2024-05-08T23:34:15.839 INFO:tasks.workunit.client.0.smithi033.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-05-08T23:34:16.076 INFO:tasks.workunit.client.0.smithi033.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-05-08T23:34:16.076 INFO:tasks.workunit.client.0.smithi033.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-05-08T23:34:16.076 INFO:tasks.workunit.client.0.smithi033.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-05-08T23:34:16.076 INFO:tasks.workunit.client.0.smithi033.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-05-08T23:34:16.208 INFO:tasks.workunit.client.0.smithi033.stdout:/bin/sh ./config.status --recheck 2024-05-08T23:34:16.437 INFO:tasks.workunit.client.0.smithi033.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-05-08T23:34:16.569 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-05-08T23:34:17.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:17.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:17.576 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether build environment is sane... yes 2024-05-08T23:34:17.582 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-05-08T23:34:17.582 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gawk... gawk 2024-05-08T23:34:17.598 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether make sets $(MAKE)... yes 2024-05-08T23:34:17.637 INFO:tasks.workunit.client.0.smithi033.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-05-08T23:34:17.637 INFO:tasks.workunit.client.0.smithi033.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-05-08T23:34:17.650 INFO:tasks.workunit.client.0.smithi033.stdout:checking for style of include used by make... GNU 2024-05-08T23:34:17.654 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... gcc 2024-05-08T23:34:18.019 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether the C compiler works... yes 2024-05-08T23:34:18.019 INFO:tasks.workunit.client.0.smithi033.stdout:checking for C compiler default output file name... a.out 2024-05-08T23:34:18.062 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of executables... 2024-05-08T23:34:18.370 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are cross compiling... no 2024-05-08T23:34:18.418 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of object files... o 2024-05-08T23:34:18.451 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T23:34:19.301 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... yes 2024-05-08T23:34:19.344 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-08T23:34:19.453 INFO:tasks.workunit.client.0.smithi033.stdout:checking dependency style of gcc... gcc3 2024-05-08T23:34:20.416 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:34:20.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:20.490 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T23:34:20.533 INFO:tasks.workunit.client.0.smithi033.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-08T23:34:20.534 INFO:tasks.workunit.client.0.smithi033.stdout:checking for egrep... /usr/bin/grep -E 2024-05-08T23:34:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.567+0000 7fa5ba655700 1 Processor -- start 2024-05-08T23:34:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.567+0000 7fa5ba655700 1 -- start start 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.568+0000 7fa5ba655700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5b415b220 0x7fa5b4157b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.568+0000 7fa5ba655700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b415ad90 con 0x7fa5b406bd10 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.568+0000 7fa5ba655700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b4071780 con 0x7fa5b40712f0 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.568+0000 7fa5ba655700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b406c800 con 0x7fa5b415b220 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa5b406bd10 0x7fa5b406c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48048/0 (socket says 172.21.15.33:48048) 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b3fff700 1 -- 172.21.15.33:0/1111723132 learned_addr learned my addr 172.21.15.33:0/1111723132 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b37fe700 1 --2- 172.21.15.33:0/1111723132 >> v2:172.21.15.33:3300/0 conn(0x7fa5b415b220 0x7fa5b4157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98187890 0 0) 0x7fa5b415ad90 con 0x7fa5b406bd10 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa59c0035b0 con 0x7fa5b406bd10 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b37fe700 1 -- 172.21.15.33:0/1111723132 >> v1:172.21.15.42:6789/0 conn(0x7fa5b40712f0 legacy=0x7fa5b4150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809145812 0 0) 0x7fa5b4071780 con 0x7fa5b40712f0 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b37fe700 1 -- 172.21.15.33:0/1111723132 >> v1:172.21.15.33:6789/0 conn(0x7fa5b406bd10 legacy=0x7fa5b406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b37fe700 1 -- 172.21.15.33:0/1111723132 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40b2fe0 con 0x7fa5b415b220 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b37fe700 1 --2- 172.21.15.33:0/1111723132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b415b220 0x7fa5b4157b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009840 tx=0x7fa5a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23272997339af21 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8011830 con 0x7fa5b415b220 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a8011990 con 0x7fa5b415b220 2024-05-08T23:34:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.569+0000 7fa5b2ffd700 1 -- 172.21.15.33:0/1111723132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8011d20 con 0x7fa5b415b220 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 -- 172.21.15.33:0/1111723132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b415b220 msgr2=0x7fa5b4157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 --2- 172.21.15.33:0/1111723132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b415b220 0x7fa5b4157b60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009840 tx=0x7fa5a8009b50 comp rx=0 tx=0).stop 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 -- 172.21.15.33:0/1111723132 shutdown_connections 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 --2- 172.21.15.33:0/1111723132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b415b220 0x7fa5b4157b60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 -- 172.21.15.33:0/1111723132 >> 172.21.15.33:0/1111723132 conn(0x7fa5b406ce60 msgr2=0x7fa5b406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 -- 172.21.15.33:0/1111723132 shutdown_connections 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.570+0000 7fa5ba655700 1 -- 172.21.15.33:0/1111723132 wait complete. 2024-05-08T23:34:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.572+0000 7fa5ba655700 1 Processor -- start 2024-05-08T23:34:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.572+0000 7fa5ba655700 1 -- start start 2024-05-08T23:34:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b406bd10 0x7fa5b41e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b415b220 0x7fa5b41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40c7c10 con 0x7fa5b40712f0 2024-05-08T23:34:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40c4390 con 0x7fa5b415b220 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.573+0000 7fa5ba655700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40c8200 con 0x7fa5b406bd10 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45062/0 (socket says 172.21.15.33:45062) 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b3fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b406bd10 0x7fa5b41e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 -- 172.21.15.33:0/157783530 learned_addr learned my addr 172.21.15.33:0/157783530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b8bf2700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b415b220 0x7fa5b41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 -- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b406bd10 msgr2=0x7fa5b41e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b406bd10 0x7fa5b41e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 -- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b415b220 msgr2=0x7fa5b41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b415b220 0x7fa5b41599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 -- 172.21.15.33:0/157783530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80094a0 con 0x7fa5b40712f0 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b37fe700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80090b0 tx=0x7fa5a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801a9f0 con 0x7fa5b40712f0 2024-05-08T23:34:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.574+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a801ab50 con 0x7fa5b40712f0 2024-05-08T23:34:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.575+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b4154480 con 0x7fa5b40712f0 2024-05-08T23:34:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.575+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80243f0 con 0x7fa5b40712f0 2024-05-08T23:34:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.575+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4154a00 con 0x7fa5b40712f0 2024-05-08T23:34:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.576+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b4073770 con 0x7fa5b40712f0 2024-05-08T23:34:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.576+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa5a8024550 con 0x7fa5b40712f0 2024-05-08T23:34:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.577+0000 7fa5b0ff9700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 0x7fa5a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.577+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa5a808a8e0 con 0x7fa5b40712f0 2024-05-08T23:34:20.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.577+0000 7fa5b3fff700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 0x7fa5a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.577+0000 7fa5b3fff700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 0x7fa5a006d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4007fd0 tx=0x7fa5a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.582+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa5a8055b10 con 0x7fa5b40712f0 2024-05-08T23:34:20.659 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ANSI C header files... yes 2024-05-08T23:34:20.694 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/types.h... yes 2024-05-08T23:34:20.734 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/stat.h... yes 2024-05-08T23:34:20.774 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdlib.h... yes 2024-05-08T23:34:20.815 INFO:tasks.workunit.client.0.smithi033.stdout:checking for string.h... yes 2024-05-08T23:34:20.857 INFO:tasks.workunit.client.0.smithi033.stdout:checking for memory.h... yes 2024-05-08T23:34:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.927+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa5b4075460 con 0x7fa5b40712f0 2024-05-08T23:34:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.927+0000 7fa5b0ff9700 1 -- 172.21.15.33:0/157783530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fa5a8015070 con 0x7fa5b40712f0 2024-05-08T23:34:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 msgr2=0x7fa5a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 0x7fa5a006d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4007fd0 tx=0x7fa5a4009040 comp rx=0 tx=0).stop 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 msgr2=0x7fa5b4153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80090b0 tx=0x7fa5a8004750 comp rx=0 tx=0).stop 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 shutdown_connections 2024-05-08T23:34:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5a006ac90 0x7fa5a006d150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b415b220 0x7fa5b41599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b406bd10 0x7fa5b41e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 --2- 172.21.15.33:0/157783530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b40712f0 0x7fa5b4153490 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 >> 172.21.15.33:0/157783530 conn(0x7fa5b406ce60 msgr2=0x7fa5b4150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 shutdown_connections 2024-05-08T23:34:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:20.930+0000 7fa5ba655700 1 -- 172.21.15.33:0/157783530 wait complete. 2024-05-08T23:34:20.943 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:34:20.943 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:34:20.943 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:34:20.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:34:20.993 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strings.h... yes 2024-05-08T23:34:21.036 INFO:tasks.workunit.client.0.smithi033.stdout:checking for inttypes.h... yes 2024-05-08T23:34:21.078 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdint.h... yes 2024-05-08T23:34:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.086+0000 7f989eccb700 1 Processor -- start 2024-05-08T23:34:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.087+0000 7f989eccb700 1 -- start start 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.087+0000 7f989eccb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f989815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.087+0000 7f989eccb700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98980c2860 con 0x7f989806d980 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.087+0000 7f989eccb700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98980c8620 con 0x7f9898154540 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.087+0000 7f989eccb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98980c3540 con 0x7f98980712f0 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f989d268700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f989806d980 0x7f989815de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48056/0 (socket says 172.21.15.33:48056) 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f989d268700 1 -- 172.21.15.33:0/3222444669 learned_addr learned my addr 172.21.15.33:0/3222444669 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f9897fff700 1 --2- 172.21.15.33:0/3222444669 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f989815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363681411 0 0) 0x7f98980c2860 con 0x7f989806d980 2024-05-08T23:34:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98880035b0 con 0x7f989806d980 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294235189 0 0) 0x7f98980c8620 con 0x7f9898154540 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98980c2860 con 0x7f9898154540 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f9897fff700 1 -- 172.21.15.33:0/3222444669 >> v1:172.21.15.42:6789/0 conn(0x7f9898154540 legacy=0x7f98980c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f9897fff700 1 -- 172.21.15.33:0/3222444669 >> v1:172.21.15.33:6789/0 conn(0x7f989806d980 legacy=0x7f989815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f9897fff700 1 -- 172.21.15.33:0/3222444669 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98980c2ae0 con 0x7f98980712f0 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.088+0000 7f9897fff700 1 --2- 172.21.15.33:0/3222444669 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f989815d8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f9884009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76be87efd2a4dd6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884011830 con 0x7f98980712f0 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9884011990 con 0x7f98980712f0 2024-05-08T23:34:21.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f98977fe700 1 -- 172.21.15.33:0/3222444669 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884011d20 con 0x7f98980712f0 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 -- 172.21.15.33:0/3222444669 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 msgr2=0x7f989815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 --2- 172.21.15.33:0/3222444669 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f989815d8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f9884009b80 comp rx=0 tx=0).stop 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 -- 172.21.15.33:0/3222444669 shutdown_connections 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 --2- 172.21.15.33:0/3222444669 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f989815d8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 -- 172.21.15.33:0/3222444669 >> 172.21.15.33:0/3222444669 conn(0x7f989806ce60 msgr2=0x7f989806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 -- 172.21.15.33:0/3222444669 shutdown_connections 2024-05-08T23:34:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.089+0000 7f989eccb700 1 -- 172.21.15.33:0/3222444669 wait complete. 2024-05-08T23:34:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.091+0000 7f989eccb700 1 Processor -- start 2024-05-08T23:34:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.091+0000 7f989eccb700 1 -- start start 2024-05-08T23:34:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f989806d980 0x7f98980b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f98981484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 0x7f989806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989815e580 con 0x7f9898154540 2024-05-08T23:34:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981efdd0 con 0x7f989806d980 2024-05-08T23:34:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989eccb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98981f0070 con 0x7f98980712f0 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 0x7f989806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 0x7f989806ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45066/0 (socket says 172.21.15.33:45066) 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 learned_addr learned my addr 172.21.15.33:0/1620414204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f9897fff700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f98981484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989ca67700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f989806d980 0x7f98980b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 msgr2=0x7f98981484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f98980712f0 0x7f98981484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f989806d980 msgr2=0x7f98980b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f989806d980 0x7f98980b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840094a0 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.092+0000 7f989d268700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 0x7f989806ab20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f989000caa0 tx=0x7f989000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.093+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000a9e0 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.093+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989000ab60 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.093+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98981f0350 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.093+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890012400 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.093+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98981f0910 con 0x7f9898154540 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f989eccb700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 0x7f9898147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9898074b70 con 0x7f989806b060 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f989ca67700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 0x7f9898147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f989ca67700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98980c2dc0 con 0x7f989806b060 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f989ca67700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 0x7f9898147590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f988c00f770 tx=0x7f988c00fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.094+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f989001b3b0 con 0x7f9898154540 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f987c06ac90 0x7f987c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0052e0 con 0x7f989806b060 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9890089070 con 0x7f9898154540 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f9897fff700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f987c06ac90 0x7f987c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f988c005440 con 0x7f989806b060 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 msgr2=0x7f9898147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 0x7f9898147590 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f988c00f770 tx=0x7f988c00fb30 comp rx=0 tx=0).stop 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 msgr2=0x7f9898147590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f98957fa700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f989806b060 0x7f9898147590 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.095+0000 7f9897fff700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f987c06ac90 0x7f987c06d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.103+0000 7f989eccb700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 0x7f98981f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.103+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989815e6c0 con 0x7f9898147ad0 2024-05-08T23:34:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.104+0000 7f989ca67700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 0x7f98981f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.104+0000 7f989ca67700 1 -- 172.21.15.33:0/1620414204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f98980c8ba0 con 0x7f9898147ad0 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.104+0000 7f989ca67700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 0x7f98981f31a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f988c017000 tx=0x7f988c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.104+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c023070 con 0x7f9898147ad0 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4970 (secure 0 0 0) 0x7f988c0133f0 con 0x7f9898147ad0 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 msgr2=0x7f98981f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f98957fa700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 0x7f98981f31a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f988c017000 tx=0x7f988c001190 comp rx=0 tx=0).stop 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f98957fa700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 msgr2=0x7f98981f31a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f98957fa700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898147ad0 0x7f98981f31a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:21.104 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 16, 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.105 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.108 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.109 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.110 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.114 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:34:21.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 8 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:34:21.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:21.117 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f987c06ac90 msgr2=0x7f987c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f987c06ac90 0x7f987c06d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c040 comp rx=0 tx=0).stop 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 msgr2=0x7f989806ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 --2- 172.21.15.33:0/1620414204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9898154540 0x7f989806ab20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f989000caa0 tx=0x7f989000cdb0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 reap_dead start 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989d268700 1 -- 172.21.15.33:0/1620414204 reap_dead start 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 shutdown_connections 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 >> 172.21.15.33:0/1620414204 conn(0x7f989806ce60 msgr2=0x7f9898153020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 shutdown_connections 2024-05-08T23:34:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.105+0000 7f989eccb700 1 -- 172.21.15.33:0/1620414204 wait complete. 2024-05-08T23:34:21.119 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:21.120 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:34:21.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:34:21.123 INFO:tasks.workunit.client.0.smithi033.stdout:checking for unistd.h... yes 2024-05-08T23:34:21.170 INFO:tasks.workunit.client.0.smithi033.stdout:checking minix/config.h usability... no 2024-05-08T23:34:21.195 INFO:tasks.workunit.client.0.smithi033.stdout:checking minix/config.h presence... no 2024-05-08T23:34:21.195 INFO:tasks.workunit.client.0.smithi033.stdout:checking for minix/config.h... no 2024-05-08T23:34:21.232 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-05-08T23:34:21.232 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... (cached) gcc 2024-05-08T23:34:21.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.263+0000 7fab45410700 1 Processor -- start 2024-05-08T23:34:21.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.263+0000 7fab45410700 1 -- start start 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab45410700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fab400c38a0 0x7fab400c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab45410700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab40070450 con 0x7fab40152350 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab45410700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab40070730 con 0x7fab400c9a60 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab45410700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab400c2e90 con 0x7fab400c38a0 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab40152350 0x7fab40158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:48058/0 (socket says 172.21.15.33:48058) 2024-05-08T23:34:21.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3effd700 1 -- 172.21.15.33:0/911241872 learned_addr learned my addr 172.21.15.33:0/911241872 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3e7fc700 1 --2- 172.21.15.33:0/911241872 >> v2:172.21.15.42:3300/0 conn(0x7fab400c38a0 0x7fab400c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171291927 0 0) 0x7fab40070450 con 0x7fab40152350 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab280035b0 con 0x7fab40152350 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3e7fc700 1 -- 172.21.15.33:0/911241872 >> v1:172.21.15.42:6789/0 conn(0x7fab400c9a60 legacy=0x7fab400c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062760598 0 0) 0x7fab280035b0 con 0x7fab40152350 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3e7fc700 1 -- 172.21.15.33:0/911241872 >> v1:172.21.15.33:6789/0 conn(0x7fab40152350 legacy=0x7fab40158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.264+0000 7fab3e7fc700 1 -- 172.21.15.33:0/911241872 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40070a10 con 0x7fab400c38a0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.265+0000 7fab3e7fc700 1 --2- 172.21.15.33:0/911241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c38a0 0x7fab400c9520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab34009840 tx=0x7fab34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2804a31bf10d356 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.265+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30001670 con 0x7fab400c38a0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.265+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab34004030 con 0x7fab400c38a0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.265+0000 7fab3dffb700 1 -- 172.21.15.33:0/911241872 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340043a0 con 0x7fab400c38a0 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 -- 172.21.15.33:0/911241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c38a0 msgr2=0x7fab400c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 --2- 172.21.15.33:0/911241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c38a0 0x7fab400c9520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab34009840 tx=0x7fab34009b50 comp rx=0 tx=0).stop 2024-05-08T23:34:21.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 -- 172.21.15.33:0/911241872 shutdown_connections 2024-05-08T23:34:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 --2- 172.21.15.33:0/911241872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c38a0 0x7fab400c9520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 -- 172.21.15.33:0/911241872 >> 172.21.15.33:0/911241872 conn(0x7fab400666c0 msgr2=0x7fab40066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 -- 172.21.15.33:0/911241872 shutdown_connections 2024-05-08T23:34:21.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.266+0000 7fab45410700 1 -- 172.21.15.33:0/911241872 wait complete. 2024-05-08T23:34:21.267 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-08T23:34:21.268 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-08T23:34:21.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.267+0000 7fab45410700 1 Processor -- start 2024-05-08T23:34:21.268 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-08T23:34:21.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.267+0000 7fab45410700 1 -- start start 2024-05-08T23:34:21.270 INFO:tasks.workunit.client.0.smithi033.stdout:checking dependency style of gcc... (cached) gcc3 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab400c38a0 0x7fab4014c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c9a60 0x7fab401ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 0x7fab401547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400b3d40 con 0x7fab40152350 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab401ed860 con 0x7fab400c9a60 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab45410700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab401587c0 con 0x7fab400c38a0 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 0x7fab401547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 0x7fab401547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45098/0 (socket says 172.21.15.33:45098) 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 learned_addr learned my addr 172.21.15.33:0/2053177759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab3e7fc700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c9a60 0x7fab401ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.268+0000 7fab3effd700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab400c38a0 0x7fab4014c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab400c38a0 msgr2=0x7fab4014c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab400c38a0 0x7fab4014c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c9a60 msgr2=0x7fab401ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab400c9a60 0x7fab401ee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340094a0 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab3f7fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 0x7fab401547a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fab3800cc50 tx=0x7fab3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38003c00 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38003d60 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab40158aa0 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3800aab0 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.269+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab401edb20 con 0x7fab40152350 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.270+0000 7fab45410700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 0x7fab40144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.270+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab401ed9d0 con 0x7fab40152a60 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.270+0000 7fab3effd700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 0x7fab40144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab3effd700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab40070110 con 0x7fab40152a60 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab3effd700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 0x7fab40144340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab3000f700 tx=0x7fab3000fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab3800ac10 con 0x7fab40152350 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab277fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab2c06ac90 0x7fab2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30011580 con 0x7fab40152a60 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fab38089380 con 0x7fab40152350 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.271+0000 7fab3e7fc700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab2c06ac90 0x7fab2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fab300116e0 con 0x7fab40152a60 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 msgr2=0x7fab40144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab277fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 0x7fab40144340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab3000f700 tx=0x7fab3000fac0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 msgr2=0x7fab40144340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab277fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab40152a60 0x7fab40144340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.272+0000 7fab3e7fc700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab2c06ac90 0x7fab2c06d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab340090b0 tx=0x7fab34008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.280+0000 7fab45410700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 0x7fab40144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.280+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4006b420 con 0x7fab401edcb0 2024-05-08T23:34:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.280+0000 7fab3effd700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 0x7fab40144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.280+0000 7fab3effd700 1 -- 172.21.15.33:0/2053177759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab400c3380 con 0x7fab401edcb0 2024-05-08T23:34:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab3effd700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 0x7fab40144880 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab30001670 tx=0x7fab30000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30022070 con 0x7fab401edcb0 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4771 (secure 0 0 0) 0x7fab30014450 con 0x7fab401edcb0 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 msgr2=0x7fab40144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 0x7fab40144880 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab30001670 tx=0x7fab30000f40 comp rx=0 tx=0).stop 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 msgr2=0x7fab40144880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.281+0000 7fab277fe700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab401edcb0 0x7fab40144880 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:21.280 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 16, 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.281 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.282 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.284 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.284 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.284 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.284 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.284 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:21.285 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:21.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:21.288 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.290 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:21.290 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:21.292 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:21.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:21.297 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.301 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.302 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.303 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:21.303 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab2c06ac90 msgr2=0x7fab2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab2c06ac90 0x7fab2c06d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab340090b0 tx=0x7fab34008030 comp rx=0 tx=0).stop 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 msgr2=0x7fab401547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 --2- 172.21.15.33:0/2053177759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab40152350 0x7fab401547a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fab3800cc50 tx=0x7fab3800cf60 comp rx=0 tx=0).stop 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 reap_dead start 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab3f7fe700 1 -- 172.21.15.33:0/2053177759 reap_dead start 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 shutdown_connections 2024-05-08T23:34:21.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 >> 172.21.15.33:0/2053177759 conn(0x7fab400666c0 msgr2=0x7fab40066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 shutdown_connections 2024-05-08T23:34:21.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.282+0000 7fab45410700 1 -- 172.21.15.33:0/2053177759 wait complete. 2024-05-08T23:34:21.307 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:21.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:34:21.308 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:34:21.312 INFO:tasks.workunit.client.0.smithi033.stdout:checking for main in -lm... yes 2024-05-08T23:34:21.359 INFO:tasks.workunit.client.0.smithi033.stdout:checking for main in -lpthread... yes 2024-05-08T23:34:21.361 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ANSI C header files... (cached) yes 2024-05-08T23:34:21.447 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-05-08T23:34:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.463+0000 7f5a6f008700 1 Processor -- start 2024-05-08T23:34:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.463+0000 7f5a6f008700 1 -- start start 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6f008700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5a6815b220 0x7f5a68157b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6f008700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680712f0 0x7f5a68150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6f008700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6815ad90 con 0x7f5a6806bd10 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6f008700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a68071780 con 0x7f5a6815b220 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6f008700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6806c800 con 0x7f5a680712f0 2024-05-08T23:34:21.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a67fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5a6815b220 0x7f5a68157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a67fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5a6815b220 0x7f5a68157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45104/0 (socket says 172.21.15.33:45104) 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a67fff700 1 -- 172.21.15.33:0/3219085213 learned_addr learned my addr 172.21.15.33:0/3219085213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.464+0000 7f5a6d5a5700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680712f0 0x7f5a68150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a67fff700 1 -- 172.21.15.33:0/3219085213 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680712f0 msgr2=0x7f5a68150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a67fff700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680712f0 0x7f5a68150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a67fff700 1 -- 172.21.15.33:0/3219085213 >> v1:172.21.15.42:6789/0 conn(0x7f5a6806bd10 legacy=0x7f5a6806c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a677fe700 1 -- 172.21.15.33:0/3219085213 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908696479 0 0) 0x7f5a6815ad90 con 0x7f5a6806bd10 2024-05-08T23:34:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a67fff700 1 -- 172.21.15.33:0/3219085213 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680b2fe0 con 0x7f5a6815b220 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a67fff700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68157b60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a540098f0 tx=0x7f5a54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2064ad3ce1c0ad83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a677fe700 1 -- 172.21.15.33:0/3219085213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400f8a0 con 0x7f5a6815b220 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a677fe700 1 -- 172.21.15.33:0/3219085213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5400fa00 con 0x7f5a6815b220 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.465+0000 7f5a677fe700 1 -- 172.21.15.33:0/3219085213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400fd90 con 0x7f5a6815b220 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 -- 172.21.15.33:0/3219085213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 msgr2=0x7f5a68157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68157b60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a540098f0 tx=0x7f5a54009c00 comp rx=0 tx=0).stop 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 -- 172.21.15.33:0/3219085213 shutdown_connections 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680712f0 0x7f5a68150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 --2- 172.21.15.33:0/3219085213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68157b60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 -- 172.21.15.33:0/3219085213 >> 172.21.15.33:0/3219085213 conn(0x7f5a6806ce60 msgr2=0x7f5a6806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 -- 172.21.15.33:0/3219085213 shutdown_connections 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.466+0000 7f5a6f008700 1 -- 172.21.15.33:0/3219085213 wait complete. 2024-05-08T23:34:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.467+0000 7f5a6f008700 1 Processor -- start 2024-05-08T23:34:21.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.467+0000 7f5a6f008700 1 -- start start 2024-05-08T23:34:21.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a6806bd10 0x7f5a68159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a680712f0 0x7f5a68147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680af8f0 con 0x7f5a6815b220 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680aea00 con 0x7f5a680712f0 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6f008700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680aece0 con 0x7f5a6806bd10 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6d5a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a67fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a680712f0 0x7f5a68147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6d5a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45106/0 (socket says 172.21.15.33:45106) 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6cda4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a6806bd10 0x7f5a68159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.468+0000 7f5a6d5a5700 1 -- 172.21.15.33:0/2165737767 learned_addr learned my addr 172.21.15.33:0/2165737767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a6806bd10 msgr2=0x7f5a68159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a6806bd10 0x7f5a68159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a680712f0 msgr2=0x7f5a68147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a680712f0 0x7f5a68147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009550 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6cda4700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a6806bd10 0x7f5a68159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a67fff700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a680712f0 0x7f5a68147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6d5a5700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000efc0 tx=0x7f5a6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60009ea0 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000f040 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a680aefc0 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000ced0 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.469+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a680af580 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.470+0000 7f5a6f008700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 0x7f5a680b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.470+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68074b70 con 0x7f5a680aff70 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.471+0000 7f5a6cda4700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 0x7f5a680b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.471+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5a60014480 con 0x7f5a6815b220 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.471+0000 7f5a6cda4700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a6806e570 con 0x7f5a680aff70 2024-05-08T23:34:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.471+0000 7f5a6cda4700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 0x7f5a680b0350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007fd0 tx=0x7f5a5c00fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.471+0000 7f5a657fa700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5a4c06acc0 0x7f5a4c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f5a60089c40 con 0x7f5a6815b220 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a67fff700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5a4c06acc0 0x7f5a4c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0052e0 con 0x7f5a680aff70 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5a5c005440 con 0x7f5a680aff70 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 msgr2=0x7f5a680b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 0x7f5a680b0350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007fd0 tx=0x7f5a5c00fe70 comp rx=0 tx=0).stop 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 msgr2=0x7f5a680b0350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a657fa700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a680aff70 0x7f5a680b0350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.472+0000 7f5a67fff700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5a4c06acc0 0x7f5a4c06d180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009c40 tx=0x7f5a5401b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.480+0000 7f5a6f008700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 0x7f5a681f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.480+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680af770 con 0x7f5a681f15a0 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.480+0000 7f5a6cda4700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 0x7f5a681f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.480+0000 7f5a6cda4700 1 -- 172.21.15.33:0/2165737767 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a680c7970 con 0x7f5a681f15a0 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.480+0000 7f5a6cda4700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 0x7f5a681f19b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007fd0 tx=0x7f5a5c001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c022070 con 0x7f5a681f15a0 2024-05-08T23:34:21.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4458 (secure 0 0 0) 0x7f5a5c01d070 con 0x7f5a681f15a0 2024-05-08T23:34:21.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 msgr2=0x7f5a681f19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 0x7f5a681f19b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c007fd0 tx=0x7f5a5c001080 comp rx=0 tx=0).stop 2024-05-08T23:34:21.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 msgr2=0x7f5a681f19b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:21.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a657fa700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f5a681f15a0 0x7f5a681f19b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 22, 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:21.481 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 16, 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:21.482 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.483 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.484 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.484 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.484 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:21.484 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:21.484 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.485 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:21.488 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:21.489 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:21.491 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.492 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:21.493 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:21.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:21.498 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:21.501 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5a4c06acc0 msgr2=0x7f5a4c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a6f008700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f5a4c06acc0 0x7f5a4c06d180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009c40 tx=0x7f5a5401b040 comp rx=0 tx=0).stop 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 msgr2=0x7f5a68147cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a6f008700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000efc0 tx=0x7f5a6000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.481+0000 7f5a6d5a5700 1 -- 172.21.15.33:0/2165737767 reap_dead start 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.482+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 shutdown_connections 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.482+0000 7f5a6f008700 1 --2- 172.21.15.33:0/2165737767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a6815b220 0x7f5a68147cb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.482+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 >> 172.21.15.33:0/2165737767 conn(0x7f5a6806ce60 msgr2=0x7f5a6806f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.482+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 shutdown_connections 2024-05-08T23:34:21.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.482+0000 7f5a6f008700 1 -- 172.21.15.33:0/2165737767 wait complete. 2024-05-08T23:34:21.504 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:34:21.504 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-08T23:34:21.504 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:34:21.504 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-08T23:34:21.504 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:34:21.505 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:34:21.505 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:34:21.507 INFO:tasks.workunit.client.0.smithi033.stdout:checking pthread.h usability... yes 2024-05-08T23:34:21.520 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:34:21.520 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:34:21.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:21.531 INFO:tasks.workunit.client.0.smithi033.stdout:checking pthread.h presence... yes 2024-05-08T23:34:21.531 INFO:tasks.workunit.client.0.smithi033.stdout:checking for pthread.h... yes 2024-05-08T23:34:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.668+0000 7f0426b6a700 1 Processor -- start 2024-05-08T23:34:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 -- start start 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0420155740 0x7f04200c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 0x7f0420071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 0x7f0420158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0420072020 con 0x7f04200c68a0 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042006c260 con 0x7f0420155740 2024-05-08T23:34:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f0426b6a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042006b650 con 0x7f0420071b90 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f041ffff700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 msgr2=0x7f0420071650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.669+0000 7f041ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 0x7f0420071650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0420155740 0x7f04200c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0420155740 0x7f04200c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46988/0 (socket says 172.21.15.33:46988) 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 -- 172.21.15.33:0/2061019498 learned_addr learned my addr 172.21.15.33:0/2061019498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0425107700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 0x7f0420158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 -- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 msgr2=0x7f0420158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 0x7f0420158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 -- 172.21.15.33:0/2061019498 >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 msgr2=0x7f0420071650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 --2- 172.21.15.33:0/2061019498 >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 0x7f0420071650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 -- 172.21.15.33:0/2061019498 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042006b930 con 0x7f0420155740 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0425107700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 0x7f0420158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.670+0000 7f0424906700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420155740 0x7f04200c6360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04200c7110 tx=0x7f0410009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ffb71d5ea27721 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f041f7fe700 1 -- 172.21.15.33:0/2061019498 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011830 con 0x7f0420155740 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f041f7fe700 1 -- 172.21.15.33:0/2061019498 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410011990 con 0x7f0420155740 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f041f7fe700 1 -- 172.21.15.33:0/2061019498 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011d20 con 0x7f0420155740 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 -- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420155740 msgr2=0x7f04200c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420155740 0x7f04200c6360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04200c7110 tx=0x7f0410009bd0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 -- 172.21.15.33:0/2061019498 shutdown_connections 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420071b90 0x7f0420158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 --2- 172.21.15.33:0/2061019498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420155740 0x7f04200c6360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 --2- 172.21.15.33:0/2061019498 >> v2:172.21.15.33:3300/0 conn(0x7f04200c68a0 0x7f0420071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 -- 172.21.15.33:0/2061019498 >> 172.21.15.33:0/2061019498 conn(0x7f042006d2b0 msgr2=0x7f042006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 -- 172.21.15.33:0/2061019498 shutdown_connections 2024-05-08T23:34:21.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.671+0000 7f0426b6a700 1 -- 172.21.15.33:0/2061019498 wait complete. 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.673+0000 7f0426b6a700 1 Processor -- start 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.673+0000 7f0426b6a700 1 -- start start 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 0x7f04201ded00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200c03f0 con 0x7f04200c68a0 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200c0920 con 0x7f0420071b90 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f0426b6a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04201e02d0 con 0x7f0420155740 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f041ffff700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 msgr2=0x7f04201ded00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.674+0000 7f041ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 0x7f04201ded00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0425107700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:47004/0 (socket says 172.21.15.33:47004) 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0425107700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:39994/0 (socket says 172.21.15.33:39994) 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 -- 172.21.15.33:0/3370947336 learned_addr learned my addr 172.21.15.33:0/3370947336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 -- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 msgr2=0x7f04200c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 -- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 msgr2=0x7f04201ded00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 0x7f04201ded00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 -- 172.21.15.33:0/3370947336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418009a70 con 0x7f0420071b90 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0425107700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0424906700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0410014040 tx=0x7f0410004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410004550 con 0x7f0420071b90 2024-05-08T23:34:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041001e070 con 0x7f0420071b90 2024-05-08T23:34:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0410009550 con 0x7f0420071b90 2024-05-08T23:34:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100038c0 con 0x7f0420071b90 2024-05-08T23:34:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.675+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04200c0b70 con 0x7f0420071b90 2024-05-08T23:34:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.676+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0420075460 con 0x7f0420071b90 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.677+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0410003a20 con 0x7f0420071b90 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.677+0000 7f041d7fa700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 0x7f040006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.678+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f041008a230 con 0x7f0420071b90 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.678+0000 7f041ffff700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 0x7f040006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.678+0000 7f041ffff700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 0x7f040006d100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04200c2060 tx=0x7f0408009270 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:21.683+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f04100552b0 con 0x7f0420071b90 2024-05-08T23:34:21.843 INFO:tasks.workunit.client.0.smithi033.stdout:checking fcntl.h usability... yes 2024-05-08T23:34:21.861 INFO:tasks.workunit.client.0.smithi033.stdout:checking fcntl.h presence... yes 2024-05-08T23:34:21.861 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fcntl.h... yes 2024-05-08T23:34:21.916 INFO:tasks.workunit.client.0.smithi033.stdout:checking limits.h usability... yes 2024-05-08T23:34:21.939 INFO:tasks.workunit.client.0.smithi033.stdout:checking limits.h presence... yes 2024-05-08T23:34:21.939 INFO:tasks.workunit.client.0.smithi033.stdout:checking for limits.h... yes 2024-05-08T23:34:21.956 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdint.h... (cached) yes 2024-05-08T23:34:21.998 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h usability... yes 2024-05-08T23:34:22.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.014+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0420154720 con 0x7f0420071b90 2024-05-08T23:34:22.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.015+0000 7f041d7fa700 1 -- 172.21.15.33:0/3370947336 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f04100588d0 con 0x7f0420071b90 2024-05-08T23:34:22.013 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:22.016 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h presence... yes 2024-05-08T23:34:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 msgr2=0x7f040006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 0x7f040006d100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04200c2060 tx=0x7f0408009270 comp rx=0 tx=0).stop 2024-05-08T23:34:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 msgr2=0x7f04200c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0410014040 tx=0x7f0410004750 comp rx=0 tx=0).stop 2024-05-08T23:34:22.017 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/time.h... yes 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 shutdown_connections 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f040006ac40 0x7f040006d100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0420155740 0x7f04200c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0420071b90 0x7f04200c1930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 --2- 172.21.15.33:0/3370947336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04200c68a0 0x7f04201ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 >> 172.21.15.33:0/3370947336 conn(0x7f042006d2b0 msgr2=0x7f0420154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 shutdown_connections 2024-05-08T23:34:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:22.018+0000 7f0426b6a700 1 -- 172.21.15.33:0/3370947336 wait complete. 2024-05-08T23:34:22.033 INFO:tasks.workunit.client.0.smithi033.stdout:checking for unistd.h... (cached) yes 2024-05-08T23:34:22.313 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/vfs.h usability... yes 2024-05-08T23:34:22.330 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/vfs.h presence... yes 2024-05-08T23:34:22.330 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/vfs.h... yes 2024-05-08T23:34:22.397 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/limits.h usability... no 2024-05-08T23:34:23.308 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/limits.h presence... no 2024-05-08T23:34:23.308 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/limits.h... no 2024-05-08T23:34:23.339 INFO:tasks.workunit.client.0.smithi033.stdout:checking for an ANSI C-conforming const... yes 2024-05-08T23:34:23.372 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-05-08T23:34:23.444 INFO:tasks.workunit.client.0.smithi033.stdout:checking for system... yes 2024-05-08T23:34:23.508 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gettimeofday... yes 2024-05-08T23:34:23.571 INFO:tasks.workunit.client.0.smithi033.stdout:checking for mkdir... yes 2024-05-08T23:34:23.636 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strerror... yes 2024-05-08T23:34:23.700 INFO:tasks.workunit.client.0.smithi033.stdout:checking for open64... yes 2024-05-08T23:34:23.764 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stat64... yes 2024-05-08T23:34:23.826 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fseeko64... yes 2024-05-08T23:34:23.889 INFO:tasks.workunit.client.0.smithi033.stdout:checking for lrand48_r... yes 2024-05-08T23:34:23.951 INFO:tasks.workunit.client.0.smithi033.stdout:checking for srand48_r... yes 2024-05-08T23:34:23.977 INFO:tasks.workunit.client.0.smithi033.stdout:configure: creating ./config.status 2024-05-08T23:34:24.044 INFO:tasks.workunit.client.0.smithi033.stdout: /bin/sh ./config.status 2024-05-08T23:34:24.100 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating Makefile 2024-05-08T23:34:24.114 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating config.h 2024-05-08T23:34:24.139 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: executing depfiles commands 2024-05-08T23:34:24.154 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: executing default commands 2024-05-08T23:34:24.180 INFO:tasks.workunit.client.0.smithi033.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-05-08T23:34:24.336 INFO:tasks.workunit.client.0.smithi033.stdout:rm -f stamp-h1 2024-05-08T23:34:24.338 INFO:tasks.workunit.client.0.smithi033.stdout:touch config.h.in 2024-05-08T23:34:24.339 INFO:tasks.workunit.client.0.smithi033.stdout:cd . && /bin/sh ./config.status config.h 2024-05-08T23:34:24.407 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating config.h 2024-05-08T23:34:24.435 INFO:tasks.workunit.client.0.smithi033.stdout:make all-am 2024-05-08T23:34:24.442 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-08T23:34:24.442 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-05-08T23:34:24.597 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-05-08T23:34:24.600 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-05-08T23:34:24.671 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-05-08T23:34:24.674 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-05-08T23:34:24.822 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-05-08T23:34:24.825 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-05-08T23:34:24.922 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-05-08T23:34:24.926 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-05-08T23:34:25.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:25.042 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-05-08T23:34:25.045 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-05-08T23:34:25.098 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-05-08T23:34:25.101 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-05-08T23:34:25.145 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-05-08T23:34:25.148 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-05-08T23:34:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.171+0000 7fbf08fa8700 1 Processor -- start 2024-05-08T23:34:25.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.171+0000 7fbf08fa8700 1 -- start start 2024-05-08T23:34:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf08fa8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf08fa8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 0x7fbf040d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf08fa8700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040c5c20 con 0x7fbf04154770 2024-05-08T23:34:25.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf08fa8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040b30b0 con 0x7fbf040d00b0 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf08fa8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf04154b50 con 0x7fbf040c4c70 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf02d9d700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 msgr2=0x7fbf040d0540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf02d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 0x7fbf040d0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf0259c700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fbf04154770 0x7fbf040c60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:50394/0 (socket says 172.21.15.33:50394) 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf0259c700 1 -- 172.21.15.33:0/2646714945 learned_addr learned my addr 172.21.15.33:0/2646714945 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbefbfff700 1 --2- 172.21.15.33:0/2646714945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf01d9b700 1 -- 172.21.15.33:0/2646714945 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462384053 0 0) 0x7fbf040c5c20 con 0x7fbf04154770 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.173+0000 7fbf01d9b700 1 -- 172.21.15.33:0/2646714945 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef00035b0 con 0x7fbf04154770 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbefbfff700 1 -- 172.21.15.33:0/2646714945 >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 msgr2=0x7fbf040d0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbefbfff700 1 --2- 172.21.15.33:0/2646714945 >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 0x7fbf040d0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbefbfff700 1 -- 172.21.15.33:0/2646714945 >> v1:172.21.15.42:6789/0 conn(0x7fbf04154770 legacy=0x7fbf040c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbefbfff700 1 -- 172.21.15.33:0/2646714945 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040caed0 con 0x7fbf040c4c70 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbefbfff700 1 --2- 172.21.15.33:0/2646714945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ae50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009840 tx=0x7fbeec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8466cf8261e9d4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:25.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbf01d9b700 1 -- 172.21.15.33:0/2646714945 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec011830 con 0x7fbf040c4c70 2024-05-08T23:34:25.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbf01d9b700 1 -- 172.21.15.33:0/2646714945 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec011990 con 0x7fbf040c4c70 2024-05-08T23:34:25.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.174+0000 7fbf01d9b700 1 -- 172.21.15.33:0/2646714945 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec011d00 con 0x7fbf040c4c70 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 -- 172.21.15.33:0/2646714945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 msgr2=0x7fbf0415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/2646714945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ae50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009840 tx=0x7fbeec009b50 comp rx=0 tx=0).stop 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 -- 172.21.15.33:0/2646714945 shutdown_connections 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/2646714945 >> v2:172.21.15.33:3300/0 conn(0x7fbf040d00b0 0x7fbf040d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/2646714945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ae50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 -- 172.21.15.33:0/2646714945 >> 172.21.15.33:0/2646714945 conn(0x7fbf04066ae0 msgr2=0x7fbf04066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 -- 172.21.15.33:0/2646714945 shutdown_connections 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.175+0000 7fbf08fa8700 1 -- 172.21.15.33:0/2646714945 wait complete. 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.176+0000 7fbf08fa8700 1 Processor -- start 2024-05-08T23:34:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.176+0000 7fbf08fa8700 1 -- start start 2024-05-08T23:34:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 0x7fbf041f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 0x7fbf040ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0415b1a0 con 0x7fbf04154770 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0414e6d0 con 0x7fbf040d00b0 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf08fa8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf041f03b0 con 0x7fbf040c4c70 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf02d9d700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 msgr2=0x7fbf040ced80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf02d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 0x7fbf040ced80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf0259c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 0x7fbf041f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf0259c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:40018/0 (socket says 172.21.15.33:40018) 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbefbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 0x7fbf041f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:47008/0 (socket says 172.21.15.33:47008) 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.177+0000 7fbf0259c700 1 -- 172.21.15.33:0/804570770 learned_addr learned my addr 172.21.15.33:0/804570770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 -- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 msgr2=0x7fbf041f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 0x7fbf041f0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 -- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 msgr2=0x7fbf040ced80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 0x7fbf040ced80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 -- 172.21.15.33:0/804570770 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec0094a0 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf0259c700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c500 tx=0x7fbef400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4005d70 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef40070d0 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4007460 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0414eee0 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.178+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0414e920 con 0x7fbf040c4c70 2024-05-08T23:34:25.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.179+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf041efee0 con 0x7fbf040c4c70 2024-05-08T23:34:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.180+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbef40075c0 con 0x7fbf040c4c70 2024-05-08T23:34:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.180+0000 7fbefaffd700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 0x7fbee406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.180+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fbef4088890 con 0x7fbf040c4c70 2024-05-08T23:34:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.180+0000 7fbefbfff700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 0x7fbee406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:25.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.181+0000 7fbefbfff700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 0x7fbee406d130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:25.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.186+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbf041efee0 con 0x7fbf040c4c70 2024-05-08T23:34:25.198 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-05-08T23:34:25.201 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-05-08T23:34:25.372 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-05-08T23:34:25.375 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-05-08T23:34:25.505 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-05-08T23:34:25.508 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-05-08T23:34:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.527+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbf041efee0 con 0x7fbf040c4c70 2024-05-08T23:34:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.528+0000 7fbefaffd700 1 -- 172.21.15.33:0/804570770 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fbf041efee0 con 0x7fbf040c4c70 2024-05-08T23:34:25.527 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 msgr2=0x7fbee406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 0x7fbee406d130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbeec000c00 tx=0x7fbeec008030 comp rx=0 tx=0).stop 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 msgr2=0x7fbf0415ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbef400c500 tx=0x7fbef400c810 comp rx=0 tx=0).stop 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 shutdown_connections 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbee406ac70 0x7fbee406d130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf04154770 0x7fbf040ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbf040c4c70 0x7fbf0415ac60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 --2- 172.21.15.33:0/804570770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf040d00b0 0x7fbf041f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 >> 172.21.15.33:0/804570770 conn(0x7fbf04066ae0 msgr2=0x7fbf04153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 shutdown_connections 2024-05-08T23:34:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:25.530+0000 7fbf08fa8700 1 -- 172.21.15.33:0/804570770 wait complete. 2024-05-08T23:34:25.657 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-05-08T23:34:25.660 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-05-08T23:34:25.729 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-05-08T23:34:25.732 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-05-08T23:34:25.806 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-05-08T23:34:25.809 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-05-08T23:34:25.916 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-05-08T23:34:25.919 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-05-08T23:34:25.946 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from parser.h:22, 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: from parser.c:25: 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c: In function ‘init_filesys’: 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.947 INFO:tasks.workunit.client.0.smithi033.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.948 INFO:tasks.workunit.client.0.smithi033.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:934:3: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, &list_head->list, list) 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-08T23:34:25.949 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-08T23:34:25.950 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:941:3: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, &list_head->list, list) { 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c: In function ‘init_tg_stats’: 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-08T23:34:25.951 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.952 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:971:6: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:39:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-05-08T23:34:25.953 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: #define offsetof(type, member) ((int) &((type *)0)->member) 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: ^ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:33:28: note: in expansion of macro ‘offsetof’ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: (type *)((char *)__mptr - offsetof(type, member)); }) 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:36:2: note: in expansion of macro ‘container_of’ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: container_of(ptr, type, member) 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr:list.h:41:13: note: in expansion of macro ‘list_entry’ 2024-05-08T23:34:25.954 INFO:tasks.workunit.client.0.smithi033.stderr: pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-05-08T23:34:25.955 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~ 2024-05-08T23:34:25.955 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:980:6: note: in expansion of macro ‘list_for_each_entry’ 2024-05-08T23:34:25.955 INFO:tasks.workunit.client.0.smithi033.stderr: list_for_each_entry(tmp_list, 2024-05-08T23:34:25.955 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:26.064 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c: In function ‘set_weight’: 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:623:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy (op, tmp_config->name, len - 7); 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:620:10: note: length computed here 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr: len = strlen(tmp_config->name); 2024-05-08T23:34:26.065 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:26.069 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c: In function ‘get_weight_total’: 2024-05-08T23:34:26.069 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:645:4: warning: ‘strncpy’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-05-08T23:34:26.069 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy (op, tmp_config->name, len - 7); 2024-05-08T23:34:26.069 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:26.070 INFO:tasks.workunit.client.0.smithi033.stderr:parser.c:642:10: note: length computed here 2024-05-08T23:34:26.070 INFO:tasks.workunit.client.0.smithi033.stderr: len = strlen(tmp_config->name); 2024-05-08T23:34:26.070 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T23:34:26.205 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-05-08T23:34:26.209 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-05-08T23:34:26.268 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-05-08T23:34:26.271 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-05-08T23:34:26.364 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-05-08T23:34:26.368 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-05-08T23:34:26.399 INFO:tasks.workunit.client.0.smithi033.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-05-08T23:34:26.404 INFO:tasks.workunit.client.0.smithi033.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-05-08T23:34:26.439 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-05-08T23:34:26.443 INFO:tasks.workunit.client.0.smithi033.stdout:FFSB version 6.0-RC2 started 2024-05-08T23:34:26.443 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout:benchmark time = 300 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout:ThreadGroup 0 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout:================ 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout: num_threads = 32 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.447 INFO:tasks.workunit.client.0.smithi033.stdout: read_random = off 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: read_size = 0 (0B) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: read_blocksize = 0 (0B) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: read_skip = off 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: read_skipsize = 0 (0B) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: write_random = on 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: write_size = 5242880 (5MB) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: fsync_file = 0 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: write_blocksize = 4096 (4KB) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: wait time = 0 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: op weights 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: read = 0 (0.00%) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: readall = 0 (0.00%) 2024-05-08T23:34:26.448 INFO:tasks.workunit.client.0.smithi033.stdout: write = 1 (100.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: create = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: append = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: delete = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: metaop = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: createdir = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: stat = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: writeall = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: writeall_fsync = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: open_close = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: write_fsync = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: create_fsync = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: append_fsync = 0 (0.00%) 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.449 INFO:tasks.workunit.client.0.smithi033.stdout:FileSystem . 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout:========== 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: num_dirs = 0 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: starting files = 128 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: min file size = 104857600 (100MB) 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: max file size = 104857600 (100MB) 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: directio = off 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: alignedio = on 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: bufferedio = off 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:26.450 INFO:tasks.workunit.client.0.smithi033.stdout: aging is off 2024-05-08T23:34:28.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.690+0000 7f411fc3f700 1 Processor -- start 2024-05-08T23:34:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 -- start start 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4118154970 0x7f411806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 0x7f41180cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411815b350 con 0x7f41180caad0 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411815e8a0 con 0x7f411806b280 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411fc3f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411815a5d0 con 0x7f4118154970 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411d1da700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 msgr2=0x7f41180cb6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.691+0000 7f411e1dc700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f41180caad0 legacy=0x7f411815d9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:34:28.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d1da700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 0x7f41180cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4118154970 0x7f411806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4118154970 0x7f411806ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:36840/0 (socket says 172.21.15.33:36840) 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 -- 172.21.15.33:0/2259535747 learned_addr learned my addr 172.21.15.33:0/2259535747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 -- 172.21.15.33:0/2259535747 >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 msgr2=0x7f41180cb6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 --2- 172.21.15.33:0/2259535747 >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 0x7f41180cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 -- 172.21.15.33:0/2259535747 >> v1:172.21.15.33:6789/0 conn(0x7f41180caad0 legacy=0x7f411815d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 -- 172.21.15.33:0/2259535747 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411815e5c0 con 0x7f4118154970 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.692+0000 7f411d9db700 1 --2- 172.21.15.33:0/2259535747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f411806ad40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41180d0d00 tx=0x7f4114009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e65f1c37436bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411c9d9700 1 -- 172.21.15.33:0/2259535747 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114004230 con 0x7f4118154970 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411c9d9700 1 -- 172.21.15.33:0/2259535747 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114004390 con 0x7f4118154970 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411c9d9700 1 -- 172.21.15.33:0/2259535747 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140036a0 con 0x7f4118154970 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 -- 172.21.15.33:0/2259535747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 msgr2=0x7f411806ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:28.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 --2- 172.21.15.33:0/2259535747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f411806ad40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41180d0d00 tx=0x7f4114009bd0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 -- 172.21.15.33:0/2259535747 shutdown_connections 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 --2- 172.21.15.33:0/2259535747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f411806ad40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 --2- 172.21.15.33:0/2259535747 >> v2:172.21.15.33:3300/0 conn(0x7f411806b280 0x7f41180cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 -- 172.21.15.33:0/2259535747 >> 172.21.15.33:0/2259535747 conn(0x7f411806cd80 msgr2=0x7f411806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 -- 172.21.15.33:0/2259535747 shutdown_connections 2024-05-08T23:34:28.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.693+0000 7f411fc3f700 1 -- 172.21.15.33:0/2259535747 wait complete. 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.695+0000 7f411fc3f700 1 Processor -- start 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.695+0000 7f411fc3f700 1 -- start start 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.695+0000 7f411fc3f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411fc3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 0x7f411814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411fc3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f41181565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411fc3f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41181e62d0 con 0x7f41180caad0 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411fc3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118156120 con 0x7f4118154970 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411fc3f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118155df0 con 0x7f411806b280 2024-05-08T23:34:28.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d1da700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 msgr2=0x7f411814bef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d1da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 0x7f411814bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411e1dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f41181565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:57620/0 (socket says 172.21.15.33:57620) 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 -- 172.21.15.33:0/1694472523 learned_addr learned my addr 172.21.15.33:0/1694472523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 -- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 msgr2=0x7f41181565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f41181565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 -- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 msgr2=0x7f411814bef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 0x7f411814bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:28.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 -- 172.21.15.33:0/1694472523 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009550 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.696+0000 7f411d9db700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4114009130 tx=0x7f4114004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.697+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114003dd0 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.697+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114011630 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.697+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41181e45d0 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.697+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140119c0 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.697+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41181e4b90 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.698+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180b4450 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.698+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4114011b20 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.699+0000 7f410e7fc700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 0x7f40fc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.699+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f411408a750 con 0x7f411806b280 2024-05-08T23:34:28.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.699+0000 7f411d1da700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 0x7f40fc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:28.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.699+0000 7f411d1da700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 0x7f40fc06d100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4118152f60 tx=0x7f4108009270 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:28.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:28.704+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4114055850 con 0x7f411806b280 2024-05-08T23:34:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.039+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4118075460 con 0x7f411806b280 2024-05-08T23:34:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.040+0000 7f410e7fc700 1 -- 172.21.15.33:0/1694472523 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f4114058e70 con 0x7f411806b280 2024-05-08T23:34:29.040 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 msgr2=0x7f40fc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 0x7f40fc06d100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4118152f60 tx=0x7f4108009270 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 msgr2=0x7f411814b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4114009130 tx=0x7f4114004750 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 shutdown_connections 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f40fc06ac40 0x7f40fc06d100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4118154970 0x7f41181565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f411806b280 0x7f411814b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 --2- 172.21.15.33:0/1694472523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180caad0 0x7f411814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 >> 172.21.15.33:0/1694472523 conn(0x7f411806cd80 msgr2=0x7f4118152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 shutdown_connections 2024-05-08T23:34:29.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:29.043+0000 7f411fc3f700 1 -- 172.21.15.33:0/1694472523 wait complete. 2024-05-08T23:34:32.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:32.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.202+0000 7fa5cfa6c700 1 Processor -- start 2024-05-08T23:34:32.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.202+0000 7fa5cfa6c700 1 -- start start 2024-05-08T23:34:32.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:32.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80c4550 con 0x7fa5c813d0f0 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80c36f0 con 0x7fa5c80c73d0 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cfa6c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80c39d0 con 0x7fa5c813cd10 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cd007700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 msgr2=0x7fa5c80c6e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.203+0000 7fa5cd007700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:36860/0 (socket says 172.21.15.33:36860) 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:57624/0 (socket says 172.21.15.33:57624) 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 learned_addr learned my addr 172.21.15.33:0/1349014760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:34:32.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:32.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.204+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.404+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 msgr2=0x7fa5c80c6e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.404+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:34:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.405+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:34:32.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.805+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 msgr2=0x7fa5c80c6e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:32.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.805+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:34:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:32.806+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:34:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.606+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 msgr2=0x7fa5c80c6e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:33.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.606+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5cd808700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:34:33.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:34:33.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:33.607+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:34:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 msgr2=0x7fa5c80668f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5c813cd10 0x7fa5c80668f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 msgr2=0x7fa5c80c8b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.42:3300/0 conn(0x7fa5c80c73d0 0x7fa5c80c8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 -- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 msgr2=0x7fa5c80c6e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.203+0000 7fa5bb7fe700 1 --2- 172.21.15.33:0/1349014760 >> v2:172.21.15.33:3300/0 conn(0x7fa5c813d0f0 0x7fa5c80c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.204+0000 7fa5bb7fe700 1 -- 172.21.15.33:0/1349014760 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80c3cb0 con 0x7fa5ac0073d0 2024-05-08T23:34:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.204+0000 7fa5bb7fe700 1 -- 172.21.15.33:0/1349014760 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac00c120 con 0x7fa5ac003720 2024-05-08T23:34:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.204+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v1:172.21.15.33:6789/0 conn(0x7fa5ac0073d0 legacy=0x7fa5ac009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:34:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.405+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v1:172.21.15.33:6789/0 conn(0x7fa5ac0073d0 legacy=0x7fa5ac009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:34:35.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.806+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> v1:172.21.15.33:6789/0 conn(0x7fa5ac0073d0 legacy=0x7fa5ac009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:34:35.960 INFO:tasks.workunit.client.0.smithi033.stdout: current utilization = 0.00% 2024-05-08T23:34:35.960 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:34:35.960 INFO:tasks.workunit.client.0.smithi033.stdout:checking existing fs: . 2024-05-08T23:34:35.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.963+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953885336 0 0) 0x7fa5ac00c120 con 0x7fa5ac003720 2024-05-08T23:34:35.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.963+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b40035b0 con 0x7fa5ac003720 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1777524300 0 0) 0x7fa5b40035b0 con 0x7fa5ac003720 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 >> v1:172.21.15.33:6789/0 conn(0x7fa5ac0073d0 legacy=0x7fa5ac009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac00d320 con 0x7fa5ac003720 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fa5bc00e590 con 0x7fa5ac003720 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 >> v1:172.21.15.42:6789/0 conn(0x7fa5ac003720 legacy=0x7fa5ac005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 reap_dead start 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 0x7fa5b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5b4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 0x7fa5b400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400c220 con 0x7fa5b4005330 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400c390 con 0x7fa5b4004850 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400c550 con 0x7fa5b4003a00 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5ce009700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 msgr2=0x7fa5b400ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5ce009700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 0x7fa5b400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5b4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 0x7fa5b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 msgr2=0x7fa5b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 0x7fa5b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 msgr2=0x7fa5b400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 0x7fa5b400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 -- 172.21.15.33:0/1349014760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40035b0 con 0x7fa5b4004850 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd808700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 0x7fa5b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.966+0000 7fa5cd007700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5b4004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007e60 tx=0x7fa5b4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fea82e23ac525f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.967+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007900 con 0x7fa5b4004850 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.967+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c400cbf0 con 0x7fa5b4004850 2024-05-08T23:34:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.967+0000 7fa5cc806700 1 -- 172.21.15.33:0/1349014760 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4011420 con 0x7fa5b4004850 2024-05-08T23:34:35.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 msgr2=0x7fa5b4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:35.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5b4004cd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007e60 tx=0x7fa5b4005880 comp rx=0 tx=0).stop 2024-05-08T23:34:35.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/1349014760 shutdown_connections 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4005330 0x7fa5b400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5b4004cd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/1349014760 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4003a00 0x7fa5b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/1349014760 >> 172.21.15.33:0/1349014760 conn(0x7fa5c8067370 msgr2=0x7fa5c806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/1349014760 shutdown_connections 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.968+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/1349014760 wait complete. 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.969+0000 7fa5cfa6c700 1 Processor -- start 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 -- start start 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 0x7fa5c81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 0x7fa5c80c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8071c50 con 0x7fa5b4003a00 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8154be0 con 0x7fa5b4004850 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cfa6c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8072530 con 0x7fa5b4005330 2024-05-08T23:34:35.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cd808700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 msgr2=0x7fa5c81546a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.970+0000 7fa5cd808700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 0x7fa5c81546a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5ce009700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 0x7fa5c80c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54986/0 (socket says 172.21.15.33:54986) 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 -- 172.21.15.33:0/3859416633 learned_addr learned my addr 172.21.15.33:0/3859416633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 -- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 msgr2=0x7fa5c80c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 0x7fa5c80c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 -- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 msgr2=0x7fa5c81546a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 0x7fa5c81546a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 -- 172.21.15.33:0/3859416633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c81553b0 con 0x7fa5b4004850 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5ce009700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 0x7fa5c80c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:35.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.971+0000 7fa5cd007700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4012d20 tx=0x7fa5c4005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:35.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.972+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007e90 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.972+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4003a40 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.972+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4026470 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.972+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c814ef30 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.972+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c814f4f0 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.973+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8075460 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.974+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa5c4023070 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.974+0000 7fa5ba7fc700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 0x7fa5ac06d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.974+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa5c408b810 con 0x7fa5b4004850 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.975+0000 7fa5cd808700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 0x7fa5ac06d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:35.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.975+0000 7fa5cd808700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 0x7fa5ac06d910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc000c00 tx=0x7fa5bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:35.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:35.979+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa5c4056af0 con 0x7fa5b4004850 2024-05-08T23:34:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.352+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa5c80c92e0 con 0x7fa5b4004850 2024-05-08T23:34:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.353+0000 7fa5ba7fc700 1 -- 172.21.15.33:0/3859416633 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fa5c402b080 con 0x7fa5b4004850 2024-05-08T23:34:36.352 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 msgr2=0x7fa5ac06d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 0x7fa5ac06d910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc000c00 tx=0x7fa5bc008040 comp rx=0 tx=0).stop 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 msgr2=0x7fa5c8154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4012d20 tx=0x7fa5c4005fb0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 shutdown_connections 2024-05-08T23:34:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa5ac06b450 0x7fa5ac06d910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa5b4005330 0x7fa5c80c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5b4003a00 0x7fa5c81546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 --2- 172.21.15.33:0/3859416633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5b4004850 0x7fa5c8154e70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.355+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 >> 172.21.15.33:0/3859416633 conn(0x7fa5c8067370 msgr2=0x7fa5c8155590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.356+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 shutdown_connections 2024-05-08T23:34:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.356+0000 7fa5cfa6c700 1 -- 172.21.15.33:0/3859416633 wait complete. 2024-05-08T23:34:36.361 INFO:tasks.workunit.client.0.smithi033.stdout:recreating new fileset 2024-05-08T23:34:36.361 INFO:tasks.workunit.client.0.smithi033.stderr:opendir: No such file or directory 2024-05-08T23:34:36.370 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:34:36.370 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:34:36.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:34:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.518+0000 7f91c237a700 1 Processor -- start 2024-05-08T23:34:36.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.519+0000 7f91c237a700 1 -- start start 2024-05-08T23:34:36.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.519+0000 7f91c237a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 0x7f91bc15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.519+0000 7f91c237a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc15ad80 0x7f91bc06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.519+0000 7f91c237a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91bc06f810 0x7f91bc06aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.519+0000 7f91c237a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc15d870 con 0x7f91bc154460 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c237a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc071150 con 0x7f91bc06f810 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c237a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c7e30 con 0x7f91bc15ad80 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91bbfff700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 msgr2=0x7f91bc15dfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91bbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 0x7f91bc15dfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91bc06f810 0x7f91bc06aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91bc06f810 0x7f91bc06aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:54990/0 (socket says 172.21.15.33:54990) 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 -- 172.21.15.33:0/3940488081 learned_addr learned my addr 172.21.15.33:0/3940488081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91bb7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc15ad80 0x7f91bc06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 -- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc15ad80 msgr2=0x7f91bc06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:36.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 --2- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc15ad80 0x7f91bc06a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 -- 172.21.15.33:0/3940488081 >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 msgr2=0x7f91bc15dfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 --2- 172.21.15.33:0/3940488081 >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 0x7f91bc15dfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.520+0000 7f91c0917700 1 -- 172.21.15.33:0/3940488081 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc15b220 con 0x7f91bc06f810 2024-05-08T23:34:36.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.521+0000 7f91c0917700 1 --2- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc06f810 0x7f91bc06aed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0c3fe0 tx=0x7f91b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177ee82dbb7f010d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.521+0000 7f91baffd700 1 -- 172.21.15.33:0/3940488081 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400e070 con 0x7f91bc06f810 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.521+0000 7f91baffd700 1 -- 172.21.15.33:0/3940488081 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b400b910 con 0x7f91bc06f810 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.521+0000 7f91baffd700 1 -- 172.21.15.33:0/3940488081 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400bcc0 con 0x7f91bc06f810 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 -- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc06f810 msgr2=0x7f91bc06aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 --2- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc06f810 0x7f91bc06aed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0c3fe0 tx=0x7f91b400b330 comp rx=0 tx=0).stop 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 -- 172.21.15.33:0/3940488081 shutdown_connections 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 --2- 172.21.15.33:0/3940488081 >> v2:172.21.15.33:3300/0 conn(0x7f91bc154460 0x7f91bc15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 --2- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc06f810 0x7f91bc06aed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 --2- 172.21.15.33:0/3940488081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc15ad80 0x7f91bc06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 -- 172.21.15.33:0/3940488081 >> 172.21.15.33:0/3940488081 conn(0x7f91bc06e990 msgr2=0x7f91bc06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 -- 172.21.15.33:0/3940488081 shutdown_connections 2024-05-08T23:34:36.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.522+0000 7f91c237a700 1 -- 172.21.15.33:0/3940488081 wait complete. 2024-05-08T23:34:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.523+0000 7f91c237a700 1 Processor -- start 2024-05-08T23:34:36.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 -- start start 2024-05-08T23:34:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 0x7f91bc1f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91bc154460 0x7f91bc1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 0x7f91bc1f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc1f0930 con 0x7f91bc154460 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c4840 con 0x7f91bc15ad80 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.524+0000 7f91c237a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c4ae0 con 0x7f91bc06f810 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bb7fe700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91bc154460 msgr2=0x7f91bc1539c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91bc154460 0x7f91bc1539c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 0x7f91bc1f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91c0917700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 0x7f91bc1f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91c0917700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 0x7f91bc1f0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55006/0 (socket says 172.21.15.33:55006) 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 0x7f91bc1f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:52124/0 (socket says 172.21.15.33:52124) 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91c0917700 1 -- 172.21.15.33:0/1156382473 learned_addr learned my addr 172.21.15.33:0/1156382473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 msgr2=0x7f91bc1f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 0x7f91bc1f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91bc154460 msgr2=0x7f91bc1539c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:36.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91bc154460 0x7f91bc1539c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91bbfff700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4009b30 con 0x7f91bc06f810 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.525+0000 7f91c0917700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc15ad80 0x7f91bc1f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91bbfff700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 0x7f91bc1f03f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00b520 tx=0x7f91ac00b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003e50 con 0x7f91bc06f810 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac005070 con 0x7f91bc06f810 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac005420 con 0x7f91bc06f810 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc0c4dc0 con 0x7f91bc06f810 2024-05-08T23:34:36.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.526+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc0c5380 con 0x7f91bc06f810 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.527+0000 7f91c237a700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 0x7f91bc1f1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.527+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0749b0 con 0x7f91bc0c6e10 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.527+0000 7f91bb7fe700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 0x7f91bc1f1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.527+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f91ac005580 con 0x7f91bc06f810 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91bb7fe700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91bc071990 con 0x7f91bc0c6e10 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91b8ff9700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91a806ac40 0x7f91a806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f91ac088df0 con 0x7f91bc06f810 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91c0917700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91a806ac40 0x7f91a806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91bb7fe700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 0x7f91bc1f1490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91bc06c2b0 tx=0x7f91b000eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0004020 con 0x7f91bc0c6e10 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.528+0000 7f91c0917700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91a806ac40 0x7f91a806d100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91bc15acc0 tx=0x7f91b4009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.529+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f91b0004180 con 0x7f91bc0c6e10 2024-05-08T23:34:36.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.529+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 msgr2=0x7f91bc1f1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.529+0000 7f91b8ff9700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 0x7f91bc1f1490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91bc06c2b0 tx=0x7f91b000eb70 comp rx=0 tx=0).stop 2024-05-08T23:34:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.529+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 msgr2=0x7f91bc1f1490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.529+0000 7f91b8ff9700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc0c6e10 0x7f91bc1f1490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.542+0000 7f91c237a700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 0x7f91bc1f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.542+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c4f50 con 0x7f91bc1f2ee0 2024-05-08T23:34:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.542+0000 7f91bb7fe700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 0x7f91bc1f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91bb7fe700 1 -- 172.21.15.33:0/1156382473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91bc0716b0 con 0x7f91bc1f2ee0 2024-05-08T23:34:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91bb7fe700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 0x7f91bc1f32f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f91b0018000 tx=0x7f91b00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0005e80 con 0x7f91bc1f2ee0 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5101 (secure 0 0 0) 0x7f91b0013940 con 0x7f91bc1f2ee0 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 msgr2=0x7f91bc1f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 0x7f91bc1f32f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f91b0018000 tx=0x7f91b00047c0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 msgr2=0x7f91bc1f32f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.543+0000 7f91b8ff9700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91bc1f2ee0 0x7f91bc1f32f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:36.543 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 24, 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.545 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:36.546 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.547 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.548 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:36.551 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:34:36.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:36.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.557 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:36.557 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:36.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91a806ac40 msgr2=0x7f91a806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91a806ac40 0x7f91a806d100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91bc15acc0 tx=0x7f91b4009000 comp rx=0 tx=0).stop 2024-05-08T23:34:36.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 msgr2=0x7f91bc1f03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 --2- 172.21.15.33:0/1156382473 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91bc06f810 0x7f91bc1f03f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00b520 tx=0x7f91ac00b830 comp rx=0 tx=0).stop 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c0917700 1 -- 172.21.15.33:0/1156382473 reap_dead start 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c0917700 1 -- 172.21.15.33:0/1156382473 reap_dead start 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 shutdown_connections 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 >> 172.21.15.33:0/1156382473 conn(0x7f91bc06e990 msgr2=0x7f91bc0cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 shutdown_connections 2024-05-08T23:34:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.544+0000 7f91c237a700 1 -- 172.21.15.33:0/1156382473 wait complete. 2024-05-08T23:34:36.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:34:36.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:34:36.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:34:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.709+0000 7f0adeb12700 1 Processor -- start 2024-05-08T23:34:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.709+0000 7f0adeb12700 1 -- start start 2024-05-08T23:34:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0adeb12700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ad80c85a0 0x7f0ad80c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0adeb12700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad8070450 con 0x7f0ad8152350 2024-05-08T23:34:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0adeb12700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad8070730 con 0x7f0ad80c38a0 2024-05-08T23:34:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0adeb12700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad80c2e90 con 0x7f0ad80c85a0 2024-05-08T23:34:36.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0adc8ae700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f0ad8152350 legacy=0x7f0ad8158110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0add0af700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f0ad80c85a0 msgr2=0x7f0ad80c8a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0add0af700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0ad80c85a0 0x7f0ad80c8a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0ad7fff700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f0ad80c38a0 0x7f0ad80c9520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:36098/0 (socket says 172.21.15.33:36098) 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.711+0000 7f0ad7fff700 1 -- 172.21.15.33:0/165401506 learned_addr learned my addr 172.21.15.33:0/165401506 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898054775 0 0) 0x7f0ad8070730 con 0x7f0ad80c38a0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ac00035b0 con 0x7f0ad80c38a0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 339739099 0 0) 0x7f0ac00035b0 con 0x7f0ad80c38a0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 >> v2:172.21.15.33:3300/0 conn(0x7f0ad80c85a0 msgr2=0x7f0ad80c8a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 --2- 172.21.15.33:0/165401506 >> v2:172.21.15.33:3300/0 conn(0x7f0ad80c85a0 0x7f0ad80c8a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 >> v1:172.21.15.33:6789/0 conn(0x7f0ad8152350 legacy=0x7f0ad8158110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8070a10 con 0x7f0ad80c38a0 2024-05-08T23:34:36.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f0acc001670 con 0x7f0ad80c38a0 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 >> v1:172.21.15.42:6789/0 conn(0x7f0ad80c38a0 legacy=0x7f0ad80c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ac0003a00 0x7f0ac0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ac0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0005260 0x7f0ac000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80c2e90 con 0x7f0ac0005260 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac000c170 con 0x7f0ac0004850 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac000c330 con 0x7f0ac0003a00 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0add0af700 1 -- 172.21.15.33:0/165401506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0005260 msgr2=0x7f0ac000ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0add0af700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0005260 0x7f0ac000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ac0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.712+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ac0003a00 0x7f0ac0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 -- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ac0003a00 msgr2=0x7f0ac0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ac0003a00 0x7f0ac0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 -- 172.21.15.33:0/165401506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0005260 msgr2=0x7f0ac000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ac0005260 0x7f0ac000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 -- 172.21.15.33:0/165401506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00035b0 con 0x7f0ac0004850 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0add0af700 1 -- 172.21.15.33:0/165401506 reap_dead start 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ac0004cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c2d0 tx=0x7f0ac0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c80c4f0a201a626 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc007c10 con 0x7f0ac0004850 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc00cd40 con 0x7f0ac0004850 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.713+0000 7f0ad77fe700 1 -- 172.21.15.33:0/165401506 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc005ba0 con 0x7f0ac0004850 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 -- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 msgr2=0x7f0ac0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ac0004cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c2d0 tx=0x7f0ac0005880 comp rx=0 tx=0).stop 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 -- 172.21.15.33:0/165401506 shutdown_connections 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 --2- 172.21.15.33:0/165401506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ac0004cd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 -- 172.21.15.33:0/165401506 >> 172.21.15.33:0/165401506 conn(0x7f0ad80666c0 msgr2=0x7f0ad8066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 -- 172.21.15.33:0/165401506 shutdown_connections 2024-05-08T23:34:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.714+0000 7f0adeb12700 1 -- 172.21.15.33:0/165401506 wait complete. 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.715+0000 7f0adeb12700 1 Processor -- start 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.715+0000 7f0adeb12700 1 -- start start 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ad81f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 0x7f0ad806baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad806bfe0 0x7f0ad81531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad8154dd0 con 0x7f0ad806bfe0 2024-05-08T23:34:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad8155260 con 0x7f0ac0004850 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0adeb12700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad81f3a80 con 0x7f0ad81f35c0 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0add0af700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad806bfe0 msgr2=0x7f0ad81531c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.716+0000 7f0add0af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad806bfe0 0x7f0ad81531c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 0x7f0ad806baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0adc8ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ad81f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 0x7f0ad806baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:52134/0 (socket says 172.21.15.33:52134) 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 -- 172.21.15.33:0/3648664459 learned_addr learned my addr 172.21.15.33:0/3648664459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 msgr2=0x7f0ad81f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ad81f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad806bfe0 msgr2=0x7f0ad81531c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad806bfe0 0x7f0ad81531c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad814d630 con 0x7f0ad81f35c0 2024-05-08T23:34:36.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ac0004850 0x7f0ad81f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad7fff700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 0x7f0ad806baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c300 tx=0x7f0acc0019c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc001670 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc0070d0 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad814d910 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.717+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc007460 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.718+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad8154a10 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.718+0000 7f0adeb12700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 0x7f0ad81483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.719+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad81f3a80 con 0x7f0ad8153700 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.719+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 0x7f0ad81483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.719+0000 7f0adc8ae700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ad8070110 con 0x7f0ad8153700 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.719+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0acc0075c0 con 0x7f0ad81f35c0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 0x7f0ad81483d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c4cb0 tx=0x7f0ac800fa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ac406ac40 0x7f0ac406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0acc08a630 con 0x7f0ad81f35c0 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0add0af700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ac406ac40 0x7f0ac406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800e640 con 0x7f0ad8153700 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0ac8009ed0 con 0x7f0ad8153700 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 msgr2=0x7f0ad81483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 0x7f0ad81483d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c4cb0 tx=0x7f0ac800fa10 comp rx=0 tx=0).stop 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 msgr2=0x7f0ad81483d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.720+0000 7f0ad57fa700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8153700 0x7f0ad81483d0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c4cb0 tx=0x7f0ac800fa10 comp rx=0 tx=0).stop 2024-05-08T23:34:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.721+0000 7f0add0af700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ac406ac40 0x7f0ac406d100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c8980 tx=0x7f0ad0006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.728+0000 7f0adeb12700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 0x7f0ad81ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.728+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad806b420 con 0x7f0ad8148910 2024-05-08T23:34:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.729+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 0x7f0ad81ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.729+0000 7f0adc8ae700 1 -- 172.21.15.33:0/3648664459 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ad80c3380 con 0x7f0ad8148910 2024-05-08T23:34:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.729+0000 7f0adc8ae700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 0x7f0ad81ef8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8010010 tx=0x7f0ac8005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8019c70 con 0x7f0ad8148910 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7f0ac8025070 con 0x7f0ad8148910 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 msgr2=0x7f0ad81ef8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 0x7f0ad81ef8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8010010 tx=0x7f0ac8005e20 comp rx=0 tx=0).stop 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 msgr2=0x7f0ad81ef8a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:34:36.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0ad57fa700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad8148910 0x7f0ad81ef8a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 24, 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:34:36.730 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.732 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:34:36.733 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:34:36.734 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.737 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:34:36.738 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:34:36.743 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ac406ac40 msgr2=0x7f0ac406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ac406ac40 0x7f0ac406d100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80c8980 tx=0x7f0ad0006c80 comp rx=0 tx=0).stop 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 msgr2=0x7f0ad806baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 --2- 172.21.15.33:0/3648664459 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0ad81f35c0 0x7f0ad806baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c300 tx=0x7f0acc0019c0 comp rx=0 tx=0).stop 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0add0af700 1 -- 172.21.15.33:0/3648664459 reap_dead start 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0add0af700 1 -- 172.21.15.33:0/3648664459 reap_dead start 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 shutdown_connections 2024-05-08T23:34:36.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.730+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 >> 172.21.15.33:0/3648664459 conn(0x7f0ad80666c0 msgr2=0x7f0ad814faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.731+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 shutdown_connections 2024-05-08T23:34:36.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:36.731+0000 7f0adeb12700 1 -- 172.21.15.33:0/3648664459 wait complete. 2024-05-08T23:34:36.746 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:34:36.746 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:34:47.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:47.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:34:56.747 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:34:56.747 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-08T23:34:56.747 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:34:56.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:34:56.762 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:34:56.762 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:34:56.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:34:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.396+0000 7ffb8b13d700 1 Processor -- start 2024-05-08T23:34:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 -- start start 2024-05-08T23:34:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 0x7ffb840ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ffb8415d730 0x7ffb840c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840723b0 con 0x7ffb841542d0 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840cd190 con 0x7ffb8415d730 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.397+0000 7ffb8b13d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8415dc00 con 0x7ffb8415ade0 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb88ed9700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 msgr2=0x7ffb840ccc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb88ed9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 0x7ffb840ccc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb896da700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb896da700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:44708/0 (socket says 172.21.15.33:44708) 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb896da700 1 -- 172.21.15.33:0/3837906379 learned_addr learned my addr 172.21.15.33:0/3837906379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.398+0000 7ffb83fff700 1 --2- 172.21.15.33:0/3837906379 >> v2:172.21.15.42:3300/0 conn(0x7ffb8415d730 0x7ffb840c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 -- 172.21.15.33:0/3837906379 >> v2:172.21.15.42:3300/0 conn(0x7ffb8415d730 msgr2=0x7ffb840c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:34:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 --2- 172.21.15.33:0/3837906379 >> v2:172.21.15.42:3300/0 conn(0x7ffb8415d730 0x7ffb840c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 -- 172.21.15.33:0/3837906379 >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 msgr2=0x7ffb840ccc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:34:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 --2- 172.21.15.33:0/3837906379 >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 0x7ffb840ccc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 -- 172.21.15.33:0/3837906379 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8415dee0 con 0x7ffb8415ade0 2024-05-08T23:34:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.399+0000 7ffb896da700 1 --2- 172.21.15.33:0/3837906379 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb840c7dc0 tx=0x7ffb7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5325dd186916ce78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.400+0000 7ffb837fe700 1 -- 172.21.15.33:0/3837906379 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00e070 con 0x7ffb8415ade0 2024-05-08T23:34:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.400+0000 7ffb837fe700 1 -- 172.21.15.33:0/3837906379 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00b9c0 con 0x7ffb8415ade0 2024-05-08T23:34:57.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.400+0000 7ffb837fe700 1 -- 172.21.15.33:0/3837906379 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00bd70 con 0x7ffb8415ade0 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 -- 172.21.15.33:0/3837906379 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 msgr2=0x7ffb840c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/3837906379 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb840c7dc0 tx=0x7ffb7c00b330 comp rx=0 tx=0).stop 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 -- 172.21.15.33:0/3837906379 shutdown_connections 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/3837906379 >> v2:172.21.15.33:3300/0 conn(0x7ffb841542d0 0x7ffb840ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/3837906379 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415ade0 0x7ffb840c7210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/3837906379 >> v2:172.21.15.42:3300/0 conn(0x7ffb8415d730 0x7ffb840c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 -- 172.21.15.33:0/3837906379 >> 172.21.15.33:0/3837906379 conn(0x7ffb8406cdd0 msgr2=0x7ffb840cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:57.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 -- 172.21.15.33:0/3837906379 shutdown_connections 2024-05-08T23:34:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.401+0000 7ffb8b13d700 1 -- 172.21.15.33:0/3837906379 wait complete. 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.402+0000 7ffb8b13d700 1 Processor -- start 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.402+0000 7ffb8b13d700 1 -- start start 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 0x7ffb84158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 0x7ffb84159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 0x7ffb841f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840bd970 con 0x7ffb8415ade0 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840bdc30 con 0x7ffb841542d0 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb8b13d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840bdf10 con 0x7ffb8415d730 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb83fff700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 msgr2=0x7ffb84159420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:34:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.403+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 0x7ffb84159420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:34:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.404+0000 7ffb88ed9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 0x7ffb84158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.404+0000 7ffb88ed9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 0x7ffb84158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:44760/0 (socket says 172.21.15.33:44760) 2024-05-08T23:34:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.404+0000 7ffb88ed9700 1 -- 172.21.15.33:0/2129024825 learned_addr learned my addr 172.21.15.33:0/2129024825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:34:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.405+0000 7ffb896da700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 0x7ffb841f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 -- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 msgr2=0x7ffb84158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 0x7ffb84158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 -- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 msgr2=0x7ffb84159420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 0x7ffb84159420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 -- 172.21.15.33:0/2129024825 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c009b30 con 0x7ffb8415d730 2024-05-08T23:34:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.406+0000 7ffb896da700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 0x7ffb841f3af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c015000 tx=0x7ffb7c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.407+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0094d0 con 0x7ffb8415d730 2024-05-08T23:34:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.407+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c004250 con 0x7ffb8415d730 2024-05-08T23:34:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.407+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c004600 con 0x7ffb8415d730 2024-05-08T23:34:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.407+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb840be1f0 con 0x7ffb8415d730 2024-05-08T23:34:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.407+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb841f35d0 con 0x7ffb8415d730 2024-05-08T23:34:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.408+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb840c69c0 con 0x7ffb8415d730 2024-05-08T23:34:57.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.409+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ffb7c004760 con 0x7ffb8415d730 2024-05-08T23:34:57.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.410+0000 7ffb817fa700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 0x7ffb6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:34:57.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.410+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ffb7c08abe0 con 0x7ffb8415d730 2024-05-08T23:34:57.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.410+0000 7ffb88ed9700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 0x7ffb6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:34:57.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.410+0000 7ffb88ed9700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 0x7ffb6806d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb780094f0 tx=0x7ffb78009480 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:34:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.421+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ffb7c055f30 con 0x7ffb8415d730 2024-05-08T23:34:57.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.879+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffb84075460 con 0x7ffb8415d730 2024-05-08T23:34:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.880+0000 7ffb817fa700 1 -- 172.21.15.33:0/2129024825 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ffb7c020080 con 0x7ffb8415d730 2024-05-08T23:34:57.878 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 msgr2=0x7ffb6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 0x7ffb6806d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb780094f0 tx=0x7ffb78009480 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 msgr2=0x7ffb841f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 0x7ffb841f3af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c015000 tx=0x7ffb7c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 shutdown_connections 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ffb6806ac90 0x7ffb6806d150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ffb8415d730 0x7ffb841f3af0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb841542d0 0x7ffb84158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 --2- 172.21.15.33:0/2129024825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb8415ade0 0x7ffb84159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 >> 172.21.15.33:0/2129024825 conn(0x7ffb8406cdd0 msgr2=0x7ffb841537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 shutdown_connections 2024-05-08T23:34:57.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:34:57.884+0000 7ffb8b13d700 1 -- 172.21.15.33:0/2129024825 wait complete. 2024-05-08T23:35:00.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:35:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.050+0000 7fcc7f94a700 1 Processor -- start 2024-05-08T23:35:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.050+0000 7fcc7f94a700 1 -- start start 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7f94a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcc78154970 0x7fcc7806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7f94a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7f94a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7815b350 con 0x7fcc780caad0 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7f94a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7815e8a0 con 0x7fcc7806b280 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7f94a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7815a5d0 con 0x7fcc78154970 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47494/0 (socket says 172.21.15.33:47494) 2024-05-08T23:35:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7dee7700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcc780caad0 0x7fcc7815d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47790/0 (socket says 172.21.15.33:47790) 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 -- 172.21.15.33:0/789660240 learned_addr learned my addr 172.21.15.33:0/789660240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 -- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 msgr2=0x7fcc780cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 -- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 msgr2=0x7fcc780cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.051+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/789660240 >> v2:172.21.15.42:3300/0 conn(0x7fcc78154970 0x7fcc7806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 msgr2=0x7fcc780cb6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/789660240 >> v1:172.21.15.33:6789/0 conn(0x7fcc780caad0 legacy=0x7fcc7815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/789660240 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7815e5c0 con 0x7fcc78154970 2024-05-08T23:35:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/789660240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc78154970 0x7fcc7806ad40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc780d0d00 tx=0x7fcc74009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e011aa45cf4938a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc6ffff700 1 -- 172.21.15.33:0/789660240 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011830 con 0x7fcc78154970 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc6ffff700 1 -- 172.21.15.33:0/789660240 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc74011990 con 0x7fcc78154970 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.052+0000 7fcc6ffff700 1 -- 172.21.15.33:0/789660240 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011d20 con 0x7fcc78154970 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 -- 172.21.15.33:0/789660240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc78154970 msgr2=0x7fcc7806ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/789660240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc78154970 0x7fcc7806ad40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcc780d0d00 tx=0x7fcc74009b20 comp rx=0 tx=0).stop 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 -- 172.21.15.33:0/789660240 shutdown_connections 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/789660240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc78154970 0x7fcc7806ad40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/789660240 >> v2:172.21.15.33:3300/0 conn(0x7fcc7806b280 0x7fcc780cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 -- 172.21.15.33:0/789660240 >> 172.21.15.33:0/789660240 conn(0x7fcc7806cd80 msgr2=0x7fcc7806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 -- 172.21.15.33:0/789660240 shutdown_connections 2024-05-08T23:35:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.053+0000 7fcc7f94a700 1 -- 172.21.15.33:0/789660240 wait complete. 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.054+0000 7fcc7f94a700 1 Processor -- start 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.054+0000 7fcc7f94a700 1 -- start start 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 0x7fcc780c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc78154970 0x7fcc781e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781f1010 con 0x7fcc7806b280 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781e6000 con 0x7fcc780caad0 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7f94a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc780c6fd0 con 0x7fcc78154970 2024-05-08T23:35:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47506/0 (socket says 172.21.15.33:47506) 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/626594193 learned_addr learned my addr 172.21.15.33:0/626594193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 msgr2=0x7fcc781f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 msgr2=0x7fcc781f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 0x7fcc780c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.055+0000 7fcc7dee7700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc78154970 0x7fcc781e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc78154970 msgr2=0x7fcc781e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc78154970 0x7fcc781e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 msgr2=0x7fcc781f0ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 -- 172.21.15.33:0/626594193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740094a0 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7cee5700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 0x7fcc780c8f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009180 tx=0x7fcc6800eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800c070 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc68004020 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680043d0 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.056+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc781e62a0 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.057+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc780c7550 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.058+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc78075460 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.058+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcc680036a0 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.059+0000 7fcc6dffb700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 0x7fcc5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.059+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fcc68088d20 con 0x7fcc780caad0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.059+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 0x7fcc5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.059+0000 7fcc7d6e6700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 0x7fcc5c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc74014040 tx=0x7fcc7401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:01.064+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcc68053e40 con 0x7fcc780caad0 2024-05-08T23:35:01.156 INFO:tasks.workunit.client.0.smithi033.stdout:fs setup took 34 secs 2024-05-08T23:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.300+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcc780cb480 con 0x7fcc780caad0 2024-05-08T23:35:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.301+0000 7fcc6dffb700 1 -- 172.21.15.33:0/626594193 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fcc68057460 con 0x7fcc780caad0 2024-05-08T23:35:02.301 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 msgr2=0x7fcc5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 0x7fcc5c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc74014040 tx=0x7fcc7401c040 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 msgr2=0x7fcc780c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 0x7fcc780c8f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009180 tx=0x7fcc6800eac0 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 shutdown_connections 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc5c06ac90 0x7fcc5c06d150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcc78154970 0x7fcc781e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7806b280 0x7fcc781f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 --2- 172.21.15.33:0/626594193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc780caad0 0x7fcc780c8f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 >> 172.21.15.33:0/626594193 conn(0x7fcc7806cd80 msgr2=0x7fcc781536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:02.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 shutdown_connections 2024-05-08T23:35:02.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:02.304+0000 7fcc7f94a700 1 -- 172.21.15.33:0/626594193 wait complete. 2024-05-08T23:35:04.060 INFO:tasks.workunit.client.0.smithi033.stdout:Syncing()...2 sec 2024-05-08T23:35:04.061 INFO:tasks.workunit.client.0.smithi033.stdout:Starting Actual Benchmark At: Wed May 8 23:35:04 2024 2024-05-08T23:35:04.061 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:35:05.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:35:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.933+0000 7f6bf2a39700 1 Processor -- start 2024-05-08T23:35:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.933+0000 7f6bf2a39700 1 -- start start 2024-05-08T23:35:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bf2a39700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6bec1545c0 0x7f6bec0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bf2a39700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec0c67a0 0x7f6bec06b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bf2a39700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec06b5e0 con 0x7f6bec0c9050 2024-05-08T23:35:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bf2a39700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec072720 con 0x7f6bec1545c0 2024-05-08T23:35:05.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bf2a39700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec15a2e0 con 0x7f6bec0c67a0 2024-05-08T23:35:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6bebfff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f6bec1545c0 0x7f6bec0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6beb7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6bec0c9050 0x7f6bec06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47610/0 (socket says 172.21.15.33:47610) 2024-05-08T23:35:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.935+0000 7f6beb7fe700 1 -- 172.21.15.33:0/3591123665 learned_addr learned my addr 172.21.15.33:0/3591123665 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:35:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.937+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec0c67a0 0x7f6bec06b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.939+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327302640 0 0) 0x7f6bec06b5e0 con 0x7f6bec0c9050 2024-05-08T23:35:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.939+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd40035b0 con 0x7f6bec0c9050 2024-05-08T23:35:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585296418 0 0) 0x7f6bd40035b0 con 0x7f6bec0c9050 2024-05-08T23:35:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec0c67a0 msgr2=0x7f6bec06b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec0c67a0 0x7f6bec06b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 >> v2:172.21.15.42:3300/0 conn(0x7f6bec1545c0 msgr2=0x7f6bec0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 --2- 172.21.15.33:0/3591123665 >> v2:172.21.15.42:3300/0 conn(0x7f6bec1545c0 0x7f6bec0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec15a580 con 0x7f6bec0c9050 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f6be0001670 con 0x7f6bec0c9050 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 >> v1:172.21.15.33:6789/0 conn(0x7f6bec0c9050 legacy=0x7f6bec06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bd4005210 0x7f6bd400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec072720 con 0x7f6bd4004850 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec15a2e0 con 0x7f6bd4003a00 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd400bf50 con 0x7f6bd4005210 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec0c67a0 0x7f6bec06b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.940+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bd4005210 0x7f6bd400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bd4003a00 msgr2=0x7f6bd4003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:35:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bd4003a00 0x7f6bd4003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/3591123665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4004850 msgr2=0x7f6bd4004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4004850 0x7f6bd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/3591123665 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec15a580 con 0x7f6bd4005210 2024-05-08T23:35:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.941+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bd4005210 0x7f6bd400ba10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6bec15e970 tx=0x7f6bd4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396669627f889c26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.945+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4003c20 con 0x7f6bd4005210 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.945+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be400f040 con 0x7f6bd4005210 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.945+0000 7f6beaffd700 1 -- 172.21.15.33:0/3591123665 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400ea30 con 0x7f6bd4005210 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.946+0000 7f6bf2a39700 1 -- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bd4005210 msgr2=0x7f6bd400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.946+0000 7f6bf2a39700 1 --2- 172.21.15.33:0/3591123665 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bd4005210 0x7f6bd400ba10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6bec15e970 tx=0x7f6bd4005880 comp rx=0 tx=0).stop 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.946+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/3591123665 reap_dead start 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.946+0000 7f6bf2a39700 1 -- 172.21.15.33:0/3591123665 shutdown_connections 2024-05-08T23:35:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.946+0000 7f6bf2a39700 1 -- 172.21.15.33:0/3591123665 >> 172.21.15.33:0/3591123665 conn(0x7f6bec0c8360 msgr2=0x7f6bec0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.955+0000 7f6bf2a39700 1 -- 172.21.15.33:0/3591123665 shutdown_connections 2024-05-08T23:35:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.956+0000 7f6bf2a39700 1 -- 172.21.15.33:0/3591123665 wait complete. 2024-05-08T23:35:05.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.958+0000 7f6bf2a39700 1 Processor -- start 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.959+0000 7f6bf2a39700 1 -- start start 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4005210 0x7f6bec0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bec149900 0x7f6bec141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec154eb0 con 0x7f6bd4005210 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec145920 con 0x7f6bec149900 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.960+0000 7f6bf2a39700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec155a00 con 0x7f6bec148160 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:50636/0 (socket says 172.21.15.33:50636) 2024-05-08T23:35:05.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/1385763289 learned_addr learned my addr 172.21.15.33:0/1385763289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6beb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bec149900 0x7f6bec141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bec149900 msgr2=0x7f6bec141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:05.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bec149900 0x7f6bec141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4005210 msgr2=0x7f6bec0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4005210 0x7f6bec0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.961+0000 7f6bf0fd6700 1 -- 172.21.15.33:0/1385763289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec15e3b0 con 0x7f6bec148160 2024-05-08T23:35:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.962+0000 7f6bf0fd6700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0c9710 tx=0x7f6be4009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:05.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.962+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4004440 con 0x7f6bec148160 2024-05-08T23:35:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.962+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be40045a0 con 0x7f6bec148160 2024-05-08T23:35:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.962+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4004910 con 0x7f6bec148160 2024-05-08T23:35:05.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.962+0000 7f6bf2a39700 1 -- 172.21.15.33:0/1385763289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec15e670 con 0x7f6bec148160 2024-05-08T23:35:05.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.966+0000 7f6bf2a39700 1 -- 172.21.15.33:0/1385763289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec157e80 con 0x7f6bec148160 2024-05-08T23:35:05.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.969+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec071ae0 con 0x7f6bec148160 2024-05-08T23:35:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.970+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6be4024070 con 0x7f6bec148160 2024-05-08T23:35:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.970+0000 7f6be8ff9700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 0x7f6bd806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.971+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f6be408b6b0 con 0x7f6bec148160 2024-05-08T23:35:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.971+0000 7f6bebfff700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 0x7f6bd806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.972+0000 7f6bebfff700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 0x7f6bd806d220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006b20 tx=0x7f6bdc006ab0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:05.981+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6be4056830 con 0x7f6bec148160 2024-05-08T23:35:06.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.963+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6bec1559e0 con 0x7f6bec148160 2024-05-08T23:35:06.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.968+0000 7f6be8ff9700 1 -- 172.21.15.33:0/1385763289 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f6be4009d60 con 0x7f6bec148160 2024-05-08T23:35:06.970 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:35:06.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.972+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 msgr2=0x7f6bd806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:06.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.972+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 0x7f6bd806d220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006b20 tx=0x7f6bdc006ab0 comp rx=0 tx=0).stop 2024-05-08T23:35:06.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.973+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 msgr2=0x7f6bec141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:06.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.973+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0c9710 tx=0x7f6be4009750 comp rx=0 tx=0).stop 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 shutdown_connections 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6bd806ad60 0x7f6bd806d220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6bec148160 0x7f6bec141980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd4005210 0x7f6bec0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 --2- 172.21.15.33:0/1385763289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6bec149900 0x7f6bec141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:06.981+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 >> 172.21.15.33:0/1385763289 conn(0x7f6bec0c8360 msgr2=0x7f6bec06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:07.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.010+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 shutdown_connections 2024-05-08T23:35:07.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.010+0000 7f6bcb7fe700 1 -- 172.21.15.33:0/1385763289 wait complete. 2024-05-08T23:35:07.037 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:35:07.037 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:35:07.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.487+0000 7f7082127700 1 Processor -- start 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.487+0000 7f7082127700 1 -- start start 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.489+0000 7f7082127700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.489+0000 7f7082127700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f707c0c2770 0x7f707c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.489+0000 7f7082127700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c06ba40 con 0x7f707c06a990 2024-05-08T23:35:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.489+0000 7f7082127700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c15f180 con 0x7f707c0c2770 2024-05-08T23:35:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.489+0000 7f7082127700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c070930 con 0x7f707c1589d0 2024-05-08T23:35:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.495+0000 7f707b7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.495+0000 7f707b7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:50640/0 (socket says 172.21.15.33:50640) 2024-05-08T23:35:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.495+0000 7f707b7fe700 1 -- 172.21.15.33:0/648276483 learned_addr learned my addr 172.21.15.33:0/648276483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.496+0000 7f707affd700 1 --2- 172.21.15.33:0/648276483 >> v2:172.21.15.42:3300/0 conn(0x7f707c0c2770 0x7f707c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.499+0000 7f707b7fe700 1 -- 172.21.15.33:0/648276483 >> v2:172.21.15.42:3300/0 conn(0x7f707c0c2770 msgr2=0x7f707c0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.499+0000 7f707b7fe700 1 --2- 172.21.15.33:0/648276483 >> v2:172.21.15.42:3300/0 conn(0x7f707c0c2770 0x7f707c0c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.499+0000 7f707b7fe700 1 -- 172.21.15.33:0/648276483 >> v1:172.21.15.42:6789/0 conn(0x7f707c06a990 legacy=0x7f707c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.499+0000 7f707b7fe700 1 -- 172.21.15.33:0/648276483 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c06b4c0 con 0x7f707c1589d0 2024-05-08T23:35:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.501+0000 7f707b7fe700 1 --2- 172.21.15.33:0/648276483 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098f0 tx=0x7f706c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f1c240464a30bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:07.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.508+0000 7f707a7fc700 1 -- 172.21.15.33:0/648276483 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c004230 con 0x7f707c1589d0 2024-05-08T23:35:07.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.508+0000 7f707a7fc700 1 -- 172.21.15.33:0/648276483 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c004390 con 0x7f707c1589d0 2024-05-08T23:35:07.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.508+0000 7f707a7fc700 1 -- 172.21.15.33:0/648276483 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0036a0 con 0x7f707c1589d0 2024-05-08T23:35:07.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.510+0000 7f7082127700 1 -- 172.21.15.33:0/648276483 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 msgr2=0x7f707c0c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:07.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.510+0000 7f7082127700 1 --2- 172.21.15.33:0/648276483 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098f0 tx=0x7f706c009c00 comp rx=0 tx=0).stop 2024-05-08T23:35:07.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.515+0000 7f7082127700 1 -- 172.21.15.33:0/648276483 shutdown_connections 2024-05-08T23:35:07.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.515+0000 7f7082127700 1 --2- 172.21.15.33:0/648276483 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c1589d0 0x7f707c0c21a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:07.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.515+0000 7f7082127700 1 --2- 172.21.15.33:0/648276483 >> v2:172.21.15.42:3300/0 conn(0x7f707c0c2770 0x7f707c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:07.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.515+0000 7f7082127700 1 -- 172.21.15.33:0/648276483 >> 172.21.15.33:0/648276483 conn(0x7f707c0c8360 msgr2=0x7f707c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:07.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.517+0000 7f7082127700 1 -- 172.21.15.33:0/648276483 shutdown_connections 2024-05-08T23:35:07.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.517+0000 7f7082127700 1 -- 172.21.15.33:0/648276483 wait complete. 2024-05-08T23:35:07.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.519+0000 7f7082127700 1 Processor -- start 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.545+0000 7f7082127700 1 -- start start 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c06a990 0x7f707c143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0c2770 0x7f707c143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 0x7f707c0c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c15de30 con 0x7f707c1589d0 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c13dfc0 con 0x7f707c0c2770 2024-05-08T23:35:07.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f7082127700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c159fe0 con 0x7f707c06a990 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f707bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 0x7f707c0c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f707bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 0x7f707c0c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47352/0 (socket says 172.21.15.33:47352) 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.547+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 learned_addr learned my addr 172.21.15.33:0/2641841849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.548+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c06a990 msgr2=0x7f707c143ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.548+0000 7f707bfff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f707c06a990 0x7f707c143ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.548+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0c2770 msgr2=0x7f707c143ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:07.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.548+0000 7f707bfff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0c2770 0x7f707c143ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.548+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c009550 con 0x7f707c1589d0 2024-05-08T23:35:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.549+0000 7f707bfff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 0x7f707c0c5f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f707400b970 tx=0x7f707400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.549+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400c9b0 con 0x7f707c1589d0 2024-05-08T23:35:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.549+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707400cb10 con 0x7f707c1589d0 2024-05-08T23:35:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.549+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074014440 con 0x7f707c1589d0 2024-05-08T23:35:07.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.556+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c1467e0 con 0x7f707c1589d0 2024-05-08T23:35:07.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.556+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c159500 con 0x7f707c1589d0 2024-05-08T23:35:07.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.558+0000 7f7082127700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 0x7f707c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.558+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c074a20 con 0x7f707c15abc0 2024-05-08T23:35:07.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.566+0000 7f707b7fe700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 0x7f707c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.566+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f707401d3b0 con 0x7f707c1589d0 2024-05-08T23:35:07.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.567+0000 7f7063fff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f706406aef0 0x7f706406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.567+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7074089200 con 0x7f707c1589d0 2024-05-08T23:35:07.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.567+0000 7f707b7fe700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f707c15f650 con 0x7f707c15abc0 2024-05-08T23:35:07.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.567+0000 7f707affd700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f706406aef0 0x7f706406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.569+0000 7f707affd700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f706406aef0 0x7f706406d3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f707c15e2a0 tx=0x7f7070008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:07.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.572+0000 7f707b7fe700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 0x7f707c0c6490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098f0 tx=0x7f706c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c003e40 con 0x7f707c15abc0 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f706c0045d0 con 0x7f707c15abc0 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 msgr2=0x7f707c0c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 0x7f707c0c6490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098f0 tx=0x7f706c004750 comp rx=0 tx=0).stop 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 msgr2=0x7f707c0c6490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:07.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.573+0000 7f7063fff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15abc0 0x7f707c0c6490 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098f0 tx=0x7f706c004750 comp rx=0 tx=0).stop 2024-05-08T23:35:07.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.590+0000 7f7082127700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 0x7f707c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:07.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.590+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c06b760 con 0x7f707c15b2d0 2024-05-08T23:35:07.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.590+0000 7f707b7fe700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 0x7f707c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:07.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.592+0000 7f707b7fe700 1 -- 172.21.15.33:0/2641841849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f707c158db0 con 0x7f707c15b2d0 2024-05-08T23:35:07.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.592+0000 7f707b7fe700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 0x7f707c159720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f706c009c40 tx=0x7f706c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c02a790 con 0x7f707c15b2d0 2024-05-08T23:35:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4459 (secure 0 0 0) 0x7f706c02a8f0 con 0x7f707c15b2d0 2024-05-08T23:35:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 msgr2=0x7f707c159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 0x7f707c159720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f706c009c40 tx=0x7f706c011000 comp rx=0 tx=0).stop 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 msgr2=0x7f707c159720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.595+0000 7f7063fff700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c15b2d0 0x7f707c159720 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f706c009c40 tx=0x7f706c011000 comp rx=0 tx=0).stop 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 3, 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:07.605 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:07.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f706406aef0 msgr2=0x7f706406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:07.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f706406aef0 0x7f706406d3b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f707c15e2a0 tx=0x7f7070008040 comp rx=0 tx=0).stop 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 msgr2=0x7f707c0c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 --2- 172.21.15.33:0/2641841849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f707c1589d0 0x7f707c0c5f50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f707400b970 tx=0x7f707400bd30 comp rx=0 tx=0).stop 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 reap_dead start 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f707bfff700 1 -- 172.21.15.33:0/2641841849 reap_dead start 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 shutdown_connections 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.599+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 >> 172.21.15.33:0/2641841849 conn(0x7f707c0c8360 msgr2=0x7f707c146c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.600+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 shutdown_connections 2024-05-08T23:35:07.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:07.600+0000 7f7082127700 1 -- 172.21.15.33:0/2641841849 wait complete. 2024-05-08T23:35:07.671 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:07.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:35:07.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:35:08.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.171+0000 7f676cf7d700 1 Processor -- start 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.171+0000 7f676cf7d700 1 -- start start 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676cf7d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680c9250 0x7f67680717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676cf7d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6768071350 con 0x7f6768158b50 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676cf7d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67680b2af0 con 0x7f67680c8ba0 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676cf7d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6768158f30 con 0x7f67680c9250 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6768158b50 0x7f676806a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47616/0 (socket says 172.21.15.33:47616) 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.174+0000 7f676659c700 1 -- 172.21.15.33:0/4185476677 learned_addr learned my addr 172.21.15.33:0/4185476677 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.175+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865694323 0 0) 0x7f6768071350 con 0x7f6768158b50 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.175+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67500035b0 con 0x7f6768158b50 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608969003 0 0) 0x7f67500035b0 con 0x7f6768158b50 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680c9250 msgr2=0x7f67680717e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680c9250 0x7f67680717e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 >> v1:172.21.15.42:6789/0 conn(0x7f67680c8ba0 legacy=0x7f6768166fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676806f600 con 0x7f6768158b50 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f67600019f0 con 0x7f6768158b50 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 >> v1:172.21.15.33:6789/0 conn(0x7f6768158b50 legacy=0x7f676806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6750003a30 0x7f6750003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750004850 0x7f6750004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750005260 0x7f675000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768158f30 con 0x7f6750005260 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675002e180 con 0x7f6750004850 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.176+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675002e340 con 0x7f6750003a30 2024-05-08T23:35:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.177+0000 7f676659c700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6750003a30 0x7f6750003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.178+0000 7f6766d9d700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750005260 0x7f675000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6765d9b700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750004850 0x7f6750004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6766d9d700 1 -- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6750003a30 msgr2=0x7f6750003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6766d9d700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6750003a30 0x7f6750003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6766d9d700 1 -- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750004850 msgr2=0x7f6750004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6766d9d700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750004850 0x7f6750004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.179+0000 7f6766d9d700 1 -- 172.21.15.33:0/4185476677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500035b0 con 0x7f6750005260 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.180+0000 7f6766d9d700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750005260 0x7f675000ba10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f67580097d0 tx=0x7f6750005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d9074f0d7a2ed4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.180+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758011620 con 0x7f6750005260 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.180+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6758011780 con 0x7f6750005260 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.180+0000 7f676559a700 1 -- 172.21.15.33:0/4185476677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758011b10 con 0x7f6750005260 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.181+0000 7f676cf7d700 1 -- 172.21.15.33:0/4185476677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750005260 msgr2=0x7f675000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.181+0000 7f676cf7d700 1 --2- 172.21.15.33:0/4185476677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750005260 0x7f675000ba10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f67580097d0 tx=0x7f6750005880 comp rx=0 tx=0).stop 2024-05-08T23:35:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.181+0000 7f6766d9d700 1 -- 172.21.15.33:0/4185476677 reap_dead start 2024-05-08T23:35:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.182+0000 7f676cf7d700 1 -- 172.21.15.33:0/4185476677 shutdown_connections 2024-05-08T23:35:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.182+0000 7f676cf7d700 1 -- 172.21.15.33:0/4185476677 >> 172.21.15.33:0/4185476677 conn(0x7f676806e990 msgr2=0x7f676806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.182+0000 7f676cf7d700 1 -- 172.21.15.33:0/4185476677 shutdown_connections 2024-05-08T23:35:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.182+0000 7f676cf7d700 1 -- 172.21.15.33:0/4185476677 wait complete. 2024-05-08T23:35:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.184+0000 7f676cf7d700 1 Processor -- start 2024-05-08T23:35:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 -- start start 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750005260 0x7f6768147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 0x7f67681613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680cc2a0 0x7f67680cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681ebde0 con 0x7f6768160ea0 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676814b7a0 con 0x7f6750005260 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.185+0000 7f676cf7d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6768160880 con 0x7f67680cc2a0 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 0x7f67681613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 0x7f67681613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47398/0 (socket says 172.21.15.33:47398) 2024-05-08T23:35:08.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 -- 172.21.15.33:0/1284015907 learned_addr learned my addr 172.21.15.33:0/1284015907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f676659c700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750005260 0x7f6768147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6766d9d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680cc2a0 0x7f67680cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680cc2a0 msgr2=0x7f67680cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f67680cc2a0 0x7f67680cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750005260 msgr2=0x7f6768147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6750005260 0x7f6768147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.186+0000 7f6765d9b700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768160b60 con 0x7f6768160ea0 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.187+0000 7f6765d9b700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 0x7f67681613f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f675c00e870 tx=0x7f675c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.188+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c0094c0 con 0x7f6768160ea0 2024-05-08T23:35:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.188+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c00c640 con 0x7f6768160ea0 2024-05-08T23:35:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.188+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00c9f0 con 0x7f6768160ea0 2024-05-08T23:35:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.188+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676814ba60 con 0x7f6768160ea0 2024-05-08T23:35:08.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.188+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6768160520 con 0x7f6768160ea0 2024-05-08T23:35:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.194+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f675c00cb50 con 0x7f6768160ea0 2024-05-08T23:35:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.195+0000 7f6756ffd700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f674806ac90 0x7f674806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.195+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f675c089350 con 0x7f6768160ea0 2024-05-08T23:35:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.195+0000 7f676659c700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f674806ac90 0x7f674806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.196+0000 7f676659c700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f674806ac90 0x7f674806d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67600013d0 tx=0x7f6760008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.202+0000 7f676cf7d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 0x7f676813d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.202+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676814b910 con 0x7f67680cbec0 2024-05-08T23:35:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.203+0000 7f6766d9d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 0x7f676813d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.203+0000 7f6766d9d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6768158560 con 0x7f67680cbec0 2024-05-08T23:35:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6766d9d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 0x7f676813d6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67580097d0 tx=0x7f6758004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675801f040 con 0x7f67680cbec0 2024-05-08T23:35:08.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6758025630 con 0x7f67680cbec0 2024-05-08T23:35:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 msgr2=0x7f676813d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 0x7f676813d6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67580097d0 tx=0x7f6758004770 comp rx=0 tx=0).stop 2024-05-08T23:35:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 msgr2=0x7f676813d6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.204+0000 7f6756ffd700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67680cbec0 0x7f676813d6d0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67580097d0 tx=0x7f6758004770 comp rx=0 tx=0).stop 2024-05-08T23:35:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.221+0000 7f676cf7d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 0x7f67681ef350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.221+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768156490 con 0x7f676813dc10 2024-05-08T23:35:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.221+0000 7f6766d9d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 0x7f67681ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.222+0000 7f6766d9d700 1 -- 172.21.15.33:0/1284015907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67680b2790 con 0x7f676813dc10 2024-05-08T23:35:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.223+0000 7f6766d9d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 0x7f67681ef350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67680cbe00 tx=0x7f6758004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758035070 con 0x7f676813dc10 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5110 (secure 0 0 0) 0x7f675801ab00 con 0x7f676813dc10 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 msgr2=0x7f67681ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 0x7f67681ef350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67680cbe00 tx=0x7f6758004700 comp rx=0 tx=0).stop 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 msgr2=0x7f67681ef350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:08.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.225+0000 7f6756ffd700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f676813dc10 0x7f67681ef350 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67680cbe00 tx=0x7f6758004700 comp rx=0 tx=0).stop 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 4, 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f674806ac90 msgr2=0x7f674806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f674806ac90 0x7f674806d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67600013d0 tx=0x7f6760008040 comp rx=0 tx=0).stop 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 msgr2=0x7f67681613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 --2- 172.21.15.33:0/1284015907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6768160ea0 0x7f67681613f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f675c00e870 tx=0x7f675c00ec30 comp rx=0 tx=0).stop 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f6766d9d700 1 -- 172.21.15.33:0/1284015907 reap_dead start 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 shutdown_connections 2024-05-08T23:35:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.233+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 >> 172.21.15.33:0/1284015907 conn(0x7f676806e990 msgr2=0x7f676806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.234+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 shutdown_connections 2024-05-08T23:35:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.234+0000 7f676cf7d700 1 -- 172.21.15.33:0/1284015907 wait complete. 2024-05-08T23:35:08.268 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:08.269 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:35:08.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:35:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.681+0000 7fa62ad0b700 1 Processor -- start 2024-05-08T23:35:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.681+0000 7fa62ad0b700 1 -- start start 2024-05-08T23:35:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa62ad0b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6240c2770 0x7fa6240c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa62ad0b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62406ba40 con 0x7fa6241589d0 2024-05-08T23:35:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa62ad0b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa624070930 con 0x7fa6240c7d80 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa62ad0b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62415f180 con 0x7fa6240c2770 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa628aa7700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa6241589d0 0x7fa62406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47618/0 (socket says 172.21.15.33:47618) 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.683+0000 7fa628aa7700 1 -- 172.21.15.33:0/1087675811 learned_addr learned my addr 172.21.15.33:0/1087675811 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.684+0000 7fa623fff700 1 --2- 172.21.15.33:0/1087675811 >> v2:172.21.15.33:3300/0 conn(0x7fa6240c2770 0x7fa6240c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.685+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808939053 0 0) 0x7fa624070930 con 0x7fa6240c7d80 2024-05-08T23:35:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.685+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa60c0035b0 con 0x7fa6240c7d80 2024-05-08T23:35:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650796680 0 0) 0x7fa60c0035b0 con 0x7fa6240c7d80 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 >> v2:172.21.15.33:3300/0 conn(0x7fa6240c2770 msgr2=0x7fa6240c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 --2- 172.21.15.33:0/1087675811 >> v2:172.21.15.33:3300/0 conn(0x7fa6240c2770 0x7fa6240c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 >> v1:172.21.15.33:6789/0 conn(0x7fa6241589d0 legacy=0x7fa62406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62406b4c0 con 0x7fa6240c7d80 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fa61c0019f0 con 0x7fa6240c7d80 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 >> v1:172.21.15.42:6789/0 conn(0x7fa6240c7d80 legacy=0x7fa624070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c003a50 0x7fa60c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 0x7fa60c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c005210 0x7fa60c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c00c0e0 con 0x7fa60c005210 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c00c250 con 0x7fa60c003a50 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.686+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c00c410 con 0x7fa60c004850 2024-05-08T23:35:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.687+0000 7fa628aa7700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c003a50 0x7fa60c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.688+0000 7fa623fff700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 0x7fa60c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.689+0000 7fa623fff700 1 -- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c003a50 msgr2=0x7fa60c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.689+0000 7fa623fff700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa60c003a50 0x7fa60c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.689+0000 7fa623fff700 1 -- 172.21.15.33:0/1087675811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c005210 msgr2=0x7fa60c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.689+0000 7fa623fff700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c005210 0x7fa60c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.689+0000 7fa623fff700 1 -- 172.21.15.33:0/1087675811 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62406b4c0 con 0x7fa60c004850 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.690+0000 7fa6292a8700 1 -- 172.21.15.33:0/1087675811 reap_dead start 2024-05-08T23:35:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.690+0000 7fa623fff700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 0x7fa60c004cd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa618009b60 tx=0x7fa60c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459f1c8b0b9aaf31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.694+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800e6c0 con 0x7fa60c004850 2024-05-08T23:35:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.694+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61800e820 con 0x7fa60c004850 2024-05-08T23:35:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.694+0000 7fa6237fe700 1 -- 172.21.15.33:0/1087675811 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800eb90 con 0x7fa60c004850 2024-05-08T23:35:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 -- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 msgr2=0x7fa60c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 0x7fa60c004cd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa618009b60 tx=0x7fa60c005880 comp rx=0 tx=0).stop 2024-05-08T23:35:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 -- 172.21.15.33:0/1087675811 shutdown_connections 2024-05-08T23:35:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/1087675811 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa60c004850 0x7fa60c004cd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 -- 172.21.15.33:0/1087675811 >> 172.21.15.33:0/1087675811 conn(0x7fa6240c8360 msgr2=0x7fa6240c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 -- 172.21.15.33:0/1087675811 shutdown_connections 2024-05-08T23:35:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.696+0000 7fa62ad0b700 1 -- 172.21.15.33:0/1087675811 wait complete. 2024-05-08T23:35:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.704+0000 7fa62ad0b700 1 Processor -- start 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.704+0000 7fa62ad0b700 1 -- start start 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 0x7fa624157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6241472a0 0x7fa624162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624162570 0x7fa62415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624161d70 con 0x7fa60c004850 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624162a20 con 0x7fa6241472a0 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.705+0000 7fa62ad0b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62415f8b0 con 0x7fa624162570 2024-05-08T23:35:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.707+0000 7fa628aa7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 0x7fa624157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.707+0000 7fa628aa7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 0x7fa624157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47432/0 (socket says 172.21.15.33:47432) 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.707+0000 7fa628aa7700 1 -- 172.21.15.33:0/125296754 learned_addr learned my addr 172.21.15.33:0/125296754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624162570 msgr2=0x7fa62415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624162570 0x7fa62415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6241472a0 msgr2=0x7fa624162030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6241472a0 0x7fa624162030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62415fb90 con 0x7fa60c004850 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.708+0000 7fa628aa7700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 0x7fa624157fc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa614007970 tx=0x7fa6140016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.709+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614012560 con 0x7fa60c004850 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.709+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6140126c0 con 0x7fa60c004850 2024-05-08T23:35:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.709+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614012a70 con 0x7fa60c004850 2024-05-08T23:35:08.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.710+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62415fe70 con 0x7fa60c004850 2024-05-08T23:35:08.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.710+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa624160430 con 0x7fa60c004850 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.713+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 0x7fa62415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.713+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624074a20 con 0x7fa624147720 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.714+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa614012bd0 con 0x7fa60c004850 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.715+0000 7fa6217fa700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa60c06c980 0x7fa60c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.715+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa6140891b0 con 0x7fa60c004850 2024-05-08T23:35:08.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.716+0000 7fa6292a8700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa60c06c980 0x7fa60c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.717+0000 7fa6292a8700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa60c06c980 0x7fa60c06ee40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6240c8160 tx=0x7fa61c0018f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.719+0000 7fa623fff700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 0x7fa62415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.729+0000 7fa623fff700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa62415f650 con 0x7fa624147720 2024-05-08T23:35:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.730+0000 7fa623fff700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 0x7fa62415eb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa618009b60 tx=0x7fa6180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618021070 con 0x7fa624147720 2024-05-08T23:35:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa618004260 con 0x7fa624147720 2024-05-08T23:35:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 msgr2=0x7fa62415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 0x7fa62415eb10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa618009b60 tx=0x7fa6180047c0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 msgr2=0x7fa62415eb10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:08.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.732+0000 7fa6217fa700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624147720 0x7fa62415eb10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.755+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 0x7fa624140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.755+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624160000 con 0x7fa624140b80 2024-05-08T23:35:08.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.760+0000 7fa623fff700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 0x7fa624140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.762+0000 7fa623fff700 1 -- 172.21.15.33:0/125296754 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa62406b760 con 0x7fa624140b80 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.764+0000 7fa623fff700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 0x7fa624140f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa618000c00 tx=0x7fa618014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800e6c0 con 0x7fa624140b80 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4967 (secure 0 0 0) 0x7fa61802e300 con 0x7fa624140b80 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 msgr2=0x7fa624140f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 0x7fa624140f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa618000c00 tx=0x7fa618014000 comp rx=0 tx=0).stop 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 msgr2=0x7fa624140f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.766+0000 7fa6217fa700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa624140b80 0x7fa624140f90 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa618000c00 tx=0x7fa618014000 comp rx=0 tx=0).stop 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 5, 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.768 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:08.769 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.773 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:08.776 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:08.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:08.779 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:08.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:08.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:08.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:08.780 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:08.780 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.775+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa60c06c980 msgr2=0x7fa60c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.775+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa60c06c980 0x7fa60c06ee40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa6240c8160 tx=0x7fa61c0018f0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.775+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 msgr2=0x7fa624157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.775+0000 7fa62ad0b700 1 --2- 172.21.15.33:0/125296754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa60c004850 0x7fa624157fc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa614007970 tx=0x7fa6140016f0 comp rx=0 tx=0).stop 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.776+0000 7fa6292a8700 1 -- 172.21.15.33:0/125296754 reap_dead start 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.776+0000 7fa6292a8700 1 -- 172.21.15.33:0/125296754 reap_dead start 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.779+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 shutdown_connections 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.779+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 >> 172.21.15.33:0/125296754 conn(0x7fa6240c8360 msgr2=0x7fa624070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.781+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 shutdown_connections 2024-05-08T23:35:08.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:08.781+0000 7fa62ad0b700 1 -- 172.21.15.33:0/125296754 wait complete. 2024-05-08T23:35:08.805 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:08.805 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:35:08.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:35:09.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.227+0000 7f7c3d7d6700 1 Processor -- start 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.231+0000 7f7c3d7d6700 1 -- start start 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.232+0000 7f7c3d7d6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c380c6f50 0x7f7c3806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.232+0000 7f7c3d7d6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c3806e690 0x7f7c3815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.232+0000 7f7c3d7d6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c38066770 con 0x7f7c38158800 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.232+0000 7f7c3d7d6700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c38072630 con 0x7f7c380c6f50 2024-05-08T23:35:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.232+0000 7f7c3d7d6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3815e770 con 0x7f7c3806e690 2024-05-08T23:35:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c380c6f50 0x7f7c3806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c380c6f50 0x7f7c3806e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47438/0 (socket says 172.21.15.33:47438) 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c36ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7c38158800 0x7f7c38162640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:47626/0 (socket says 172.21.15.33:47626) 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 -- 172.21.15.33:0/4170804787 learned_addr learned my addr 172.21.15.33:0/4170804787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c377fe700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c3806e690 0x7f7c3815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 -- 172.21.15.33:0/4170804787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c3806e690 msgr2=0x7f7c3815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c3806e690 0x7f7c3815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 -- 172.21.15.33:0/4170804787 >> v1:172.21.15.33:6789/0 conn(0x7f7c38158800 legacy=0x7f7c38162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 -- 172.21.15.33:0/4170804787 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380c73b0 con 0x7f7c380c6f50 2024-05-08T23:35:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.233+0000 7f7c367fc700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c380c6f50 0x7f7c3806e150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098f0 tx=0x7f7c2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1987f01cf12a9b39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.234+0000 7f7c35ffb700 1 -- 172.21.15.33:0/4170804787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004230 con 0x7f7c380c6f50 2024-05-08T23:35:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.234+0000 7f7c35ffb700 1 -- 172.21.15.33:0/4170804787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c004390 con 0x7f7c380c6f50 2024-05-08T23:35:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.234+0000 7f7c35ffb700 1 -- 172.21.15.33:0/4170804787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0036a0 con 0x7f7c380c6f50 2024-05-08T23:35:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.235+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/4170804787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c380c6f50 msgr2=0x7f7c3806e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.235+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c380c6f50 0x7f7c3806e150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098f0 tx=0x7f7c2c009c00 comp rx=0 tx=0).stop 2024-05-08T23:35:09.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.236+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/4170804787 shutdown_connections 2024-05-08T23:35:09.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.236+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c3806e690 0x7f7c3815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:09.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.236+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/4170804787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c380c6f50 0x7f7c3806e150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:09.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.236+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/4170804787 >> 172.21.15.33:0/4170804787 conn(0x7f7c3806a990 msgr2=0x7f7c3806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.244+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/4170804787 shutdown_connections 2024-05-08T23:35:09.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.245+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/4170804787 wait complete. 2024-05-08T23:35:09.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.248+0000 7f7c3d7d6700 1 Processor -- start 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.251+0000 7f7c3d7d6700 1 -- start start 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c6f50 0x7f7c3815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 0x7f7c3813cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38143a20 con 0x7f7c381471d0 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38143ce0 con 0x7f7c3806e690 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c3d7d6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38146410 con 0x7f7c380c6f50 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 0x7f7c3813cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c36ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 0x7f7c3813cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47444/0 (socket says 172.21.15.33:47444) 2024-05-08T23:35:09.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c377fe700 1 -- 172.21.15.33:0/3349305916 learned_addr learned my addr 172.21.15.33:0/3349305916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.252+0000 7f7c36ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:60974/0 (socket says 172.21.15.33:60974) 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 -- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c6f50 msgr2=0x7f7c3815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c6f50 0x7f7c3815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 -- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 msgr2=0x7f7c3813cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 0x7f7c3813cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 -- 172.21.15.33:0/3349305916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c009550 con 0x7f7c3806e690 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.253+0000 7f7c36ffd700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c380738d0 tx=0x7f7c2800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.255+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28005cb0 con 0x7f7c3806e690 2024-05-08T23:35:09.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.255+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28005e10 con 0x7f7c3806e690 2024-05-08T23:35:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.255+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007310 con 0x7f7c3806e690 2024-05-08T23:35:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.255+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c381466f0 con 0x7f7c3806e690 2024-05-08T23:35:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.255+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38146cb0 con 0x7f7c3806e690 2024-05-08T23:35:09.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.256+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c38146ed0 con 0x7f7c3806e690 2024-05-08T23:35:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.270+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c28007470 con 0x7f7c3806e690 2024-05-08T23:35:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.271+0000 7f7c1f7fe700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 0x7f7c2406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.271+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7c28088d70 con 0x7f7c3806e690 2024-05-08T23:35:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.272+0000 7f7c367fc700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 0x7f7c2406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:09.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.272+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c280540a0 con 0x7f7c3806e690 2024-05-08T23:35:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.273+0000 7f7c367fc700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 0x7f7c2406d180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c3815b750 tx=0x7f7c2c010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:09.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:09.992+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f7c38075460 con 0x7f7c3806e690 2024-05-08T23:35:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.004+0000 7f7c1f7fe700 1 -- 172.21.15.33:0/3349305916 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f7c28021080 con 0x7f7c3806e690 2024-05-08T23:35:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 msgr2=0x7f7c2406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 0x7f7c2406d180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c3815b750 tx=0x7f7c2c010000 comp rx=0 tx=0).stop 2024-05-08T23:35:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 msgr2=0x7f7c380ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c380738d0 tx=0x7f7c2800c730 comp rx=0 tx=0).stop 2024-05-08T23:35:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 shutdown_connections 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406acc0 0x7f7c2406d180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c381471d0 0x7f7c3813cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c3806e690 0x7f7c380ca4f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 --2- 172.21.15.33:0/3349305916 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c6f50 0x7f7c3815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 >> 172.21.15.33:0/3349305916 conn(0x7f7c3806a990 msgr2=0x7f7c3815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 shutdown_connections 2024-05-08T23:35:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:10.035+0000 7f7c3d7d6700 1 -- 172.21.15.33:0/3349305916 wait complete. 2024-05-08T23:35:10.082 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:35:17.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:35:17.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:35:20.084 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:35:20.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:35:20.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.547+0000 7fc054de1700 1 Processor -- start 2024-05-08T23:35:20.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.547+0000 7fc054de1700 1 -- start start 2024-05-08T23:35:20.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.548+0000 7fc054de1700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc050161150 0x7fc0500c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.548+0000 7fc054de1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc050166e10 0x7fc0500cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.549+0000 7fc054de1700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05006bae0 con 0x7fc0500c0b10 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.549+0000 7fc054de1700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05016b250 con 0x7fc050166e10 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.549+0000 7fc054de1700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0501672c0 con 0x7fc050161150 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.550+0000 7fc04f7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc050161150 0x7fc0500c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.550+0000 7fc04f7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc050161150 0x7fc0500c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55230/0 (socket says 172.21.15.33:55230) 2024-05-08T23:35:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.550+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3049143102 learned_addr learned my addr 172.21.15.33:0/3049143102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:20.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.551+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3049143102 >> v2:172.21.15.33:3300/0 conn(0x7fc050166e10 msgr2=0x7fc0500cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:20.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.551+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3049143102 >> v2:172.21.15.33:3300/0 conn(0x7fc050166e10 0x7fc0500cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:20.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.551+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3049143102 >> v1:172.21.15.42:6789/0 conn(0x7fc0500c0b10 legacy=0x7fc050168940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:20.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.551+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3049143102 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0501675a0 con 0x7fc050161150 2024-05-08T23:35:20.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.551+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3049143102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050161150 0x7fc0500c65b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc05006b040 tx=0x7fc04400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ee4357015546f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:20.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.553+0000 7fc04e7fc700 1 -- 172.21.15.33:0/3049143102 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044010070 con 0x7fc050161150 2024-05-08T23:35:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.553+0000 7fc04e7fc700 1 -- 172.21.15.33:0/3049143102 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc044004020 con 0x7fc050161150 2024-05-08T23:35:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.553+0000 7fc04e7fc700 1 -- 172.21.15.33:0/3049143102 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440043b0 con 0x7fc050161150 2024-05-08T23:35:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.554+0000 7fc054de1700 1 -- 172.21.15.33:0/3049143102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050161150 msgr2=0x7fc0500c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:20.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.554+0000 7fc054de1700 1 --2- 172.21.15.33:0/3049143102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050161150 0x7fc0500c65b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc05006b040 tx=0x7fc04400d330 comp rx=0 tx=0).stop 2024-05-08T23:35:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 -- 172.21.15.33:0/3049143102 shutdown_connections 2024-05-08T23:35:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 --2- 172.21.15.33:0/3049143102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050161150 0x7fc0500c65b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 --2- 172.21.15.33:0/3049143102 >> v2:172.21.15.33:3300/0 conn(0x7fc050166e10 0x7fc0500cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 -- 172.21.15.33:0/3049143102 >> 172.21.15.33:0/3049143102 conn(0x7fc0500666c0 msgr2=0x7fc050066ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:20.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 -- 172.21.15.33:0/3049143102 shutdown_connections 2024-05-08T23:35:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.559+0000 7fc054de1700 1 -- 172.21.15.33:0/3049143102 wait complete. 2024-05-08T23:35:20.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.561+0000 7fc054de1700 1 Processor -- start 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.562+0000 7fc054de1700 1 -- start start 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 0x7fc05008b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc050161150 0x7fc0500c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 0x7fc0500c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc050166160 con 0x7fc050161150 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05020e2f0 con 0x7fc050166e10 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.563+0000 7fc054de1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05020e830 con 0x7fc0500c0b10 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.564+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 0x7fc0500c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.564+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 0x7fc0500c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:55234/0 (socket says 172.21.15.33:55234) 2024-05-08T23:35:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.564+0000 7fc04ffff700 1 -- 172.21.15.33:0/3733737694 learned_addr learned my addr 172.21.15.33:0/3733737694 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.565+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 0x7fc05008b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.566+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 msgr2=0x7fc0500c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.566+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 0x7fc0500c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.566+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc050161150 msgr2=0x7fc0500c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.566+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc050161150 0x7fc0500c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.566+0000 7fc04f7fe700 1 -- 172.21.15.33:0/3733737694 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440086d0 con 0x7fc0500c0b10 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.568+0000 7fc04f7fe700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 0x7fc05008b9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc044018000 tx=0x7fc044004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.568+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044008d10 con 0x7fc0500c0b10 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.569+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0500b0e70 con 0x7fc0500c0b10 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.569+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05020e550 con 0x7fc0500c0b10 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.569+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc044008e70 con 0x7fc0500c0b10 2024-05-08T23:35:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.569+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440038c0 con 0x7fc0500c0b10 2024-05-08T23:35:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.570+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc050075350 con 0x7fc0500c0b10 2024-05-08T23:35:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.572+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc044003a20 con 0x7fc0500c0b10 2024-05-08T23:35:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.572+0000 7fc03ffff700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 0x7fc03406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.573+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fc04408a830 con 0x7fc0500c0b10 2024-05-08T23:35:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.574+0000 7fc04effd700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 0x7fc03406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.575+0000 7fc04effd700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 0x7fc03406d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc050072390 tx=0x7fc040006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:20.583+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc044055b10 con 0x7fc0500c0b10 2024-05-08T23:35:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.385+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc0500c6c00 con 0x7fc0500c0b10 2024-05-08T23:35:21.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.388+0000 7fc03ffff700 1 -- 172.21.15.33:0/3733737694 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fc04401a620 con 0x7fc0500c0b10 2024-05-08T23:35:21.388 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 msgr2=0x7fc03406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 0x7fc03406d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc050072390 tx=0x7fc040006c80 comp rx=0 tx=0).stop 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 msgr2=0x7fc05008b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 0x7fc05008b9e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc044018000 tx=0x7fc044004750 comp rx=0 tx=0).stop 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 shutdown_connections 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc03406ac90 0x7fc03406d150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc050166e10 0x7fc0500c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc0500c0b10 0x7fc05008b9e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 --2- 172.21.15.33:0/3733737694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc050161150 0x7fc0500c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.392+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 >> 172.21.15.33:0/3733737694 conn(0x7fc0500666c0 msgr2=0x7fc050066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.393+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 shutdown_connections 2024-05-08T23:35:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.394+0000 7fc054de1700 1 -- 172.21.15.33:0/3733737694 wait complete. 2024-05-08T23:35:21.416 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:35:21.416 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:35:21.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:35:21.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:35:21.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.731+0000 7f48f47ac700 1 Processor -- start 2024-05-08T23:35:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.731+0000 7f48f47ac700 1 -- start start 2024-05-08T23:35:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48ec0c85c0 0x7f48ec0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec067b90 0x7f48ec1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f48ec150830 0x7f48ec1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0730d0 con 0x7f48ec0c85c0 2024-05-08T23:35:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0c2bb0 con 0x7f48ec150830 2024-05-08T23:35:21.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.733+0000 7f48f47ac700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec06cfa0 con 0x7f48ec067b90 2024-05-08T23:35:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.735+0000 7f48f2548700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48ec0c85c0 0x7f48ec0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.735+0000 7f48f2548700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48ec0c85c0 0x7f48ec0c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60046/0 (socket says 172.21.15.33:60046) 2024-05-08T23:35:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.735+0000 7f48f2548700 1 -- 172.21.15.33:0/1209938547 learned_addr learned my addr 172.21.15.33:0/1209938547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.736+0000 7f48f2d49700 1 --2- 172.21.15.33:0/1209938547 >> v2:172.21.15.42:3300/0 conn(0x7f48ec150830 0x7f48ec1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.736+0000 7f48f1d47700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec067b90 0x7f48ec1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.737+0000 7f48f2548700 1 -- 172.21.15.33:0/1209938547 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec067b90 msgr2=0x7f48ec1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.737+0000 7f48f2548700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec067b90 0x7f48ec1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.737+0000 7f48f2548700 1 -- 172.21.15.33:0/1209938547 >> v2:172.21.15.42:3300/0 conn(0x7f48ec150830 msgr2=0x7f48ec1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.737+0000 7f48f2548700 1 --2- 172.21.15.33:0/1209938547 >> v2:172.21.15.42:3300/0 conn(0x7f48ec150830 0x7f48ec1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.737+0000 7f48f2548700 1 -- 172.21.15.33:0/1209938547 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec06d280 con 0x7f48ec0c85c0 2024-05-08T23:35:21.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.738+0000 7f48f2548700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c2670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0c6580 tx=0x7f48e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2458d16874d5aa4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.739+0000 7f48f1546700 1 -- 172.21.15.33:0/1209938547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8011830 con 0x7f48ec0c85c0 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.739+0000 7f48f1546700 1 -- 172.21.15.33:0/1209938547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8011990 con 0x7f48ec0c85c0 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.739+0000 7f48f1546700 1 -- 172.21.15.33:0/1209938547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8011d20 con 0x7f48ec0c85c0 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 -- 172.21.15.33:0/1209938547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 msgr2=0x7f48ec0c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c2670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0c6580 tx=0x7f48e8009c00 comp rx=0 tx=0).stop 2024-05-08T23:35:21.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 -- 172.21.15.33:0/1209938547 shutdown_connections 2024-05-08T23:35:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c2670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 --2- 172.21.15.33:0/1209938547 >> v2:172.21.15.42:3300/0 conn(0x7f48ec150830 0x7f48ec1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 --2- 172.21.15.33:0/1209938547 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec067b90 0x7f48ec1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.740+0000 7f48f47ac700 1 -- 172.21.15.33:0/1209938547 >> 172.21.15.33:0/1209938547 conn(0x7f48ec0c89c0 msgr2=0x7f48ec0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:21.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.741+0000 7f48f47ac700 1 -- 172.21.15.33:0/1209938547 shutdown_connections 2024-05-08T23:35:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.741+0000 7f48f47ac700 1 -- 172.21.15.33:0/1209938547 wait complete. 2024-05-08T23:35:21.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.743+0000 7f48f47ac700 1 Processor -- start 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.744+0000 7f48f47ac700 1 -- start start 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48ec067b90 0x7f48ec152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec0c5ce0 0x7f48ec1f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec13f5f0 con 0x7f48ec0c85c0 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec153330 con 0x7f48ec067b90 2024-05-08T23:35:21.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.745+0000 7f48f47ac700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c32d0 con 0x7f48ec0c5ce0 2024-05-08T23:35:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60048/0 (socket says 172.21.15.33:60048) 2024-05-08T23:35:21.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 -- 172.21.15.33:0/3416467891 learned_addr learned my addr 172.21.15.33:0/3416467891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec0c5ce0 msgr2=0x7f48ec1f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48ec0c5ce0 0x7f48ec1f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48ec067b90 msgr2=0x7f48ec152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48ec067b90 0x7f48ec152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.746+0000 7f48f1d47700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8009550 con 0x7f48ec0c85c0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.747+0000 7f48f1d47700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c57a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e840 tx=0x7f48dc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.748+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc013680 con 0x7f48ec0c85c0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.748+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48dc0137e0 con 0x7f48ec0c85c0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.748+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48dc013b50 con 0x7f48ec0c85c0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.748+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0c35b0 con 0x7f48ec0c85c0 2024-05-08T23:35:21.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.748+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec0c3b70 con 0x7f48ec0c85c0 2024-05-08T23:35:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.750+0000 7f48f47ac700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 0x7f48d4004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.750+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d4007f20 con 0x7f48d4002a40 2024-05-08T23:35:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.750+0000 7f48f2548700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 0x7f48d4004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.751+0000 7f48f2548700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48d40080e0 con 0x7f48d4002a40 2024-05-08T23:35:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.751+0000 7f48f2548700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 0x7f48d4004f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48e8016000 tx=0x7f48e8012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.752+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004240 con 0x7f48d4002a40 2024-05-08T23:35:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.753+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f48dc013cb0 con 0x7f48ec0c85c0 2024-05-08T23:35:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.754+0000 7f48e2ffd700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48d006ad80 0x7f48d006d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.754+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f48dc088f70 con 0x7f48ec0c85c0 2024-05-08T23:35:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.754+0000 7f48f2d49700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48d006ad80 0x7f48d006d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.755+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f48e80043a0 con 0x7f48d4002a40 2024-05-08T23:35:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.755+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 msgr2=0x7f48d4004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.755+0000 7f48e2ffd700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 0x7f48d4004f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48e8016000 tx=0x7f48e8012df0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.755+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 msgr2=0x7f48d4004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.755+0000 7f48e2ffd700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4002a40 0x7f48d4004f00 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48e8016000 tx=0x7f48e8012df0 comp rx=0 tx=0).stop 2024-05-08T23:35:21.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.761+0000 7f48f2d49700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48d006ad80 0x7f48d006d240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0aa5f0 tx=0x7f48e400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.768+0000 7f48f47ac700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 0x7f48d40074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.768+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d400a3e0 con 0x7f48d4007020 2024-05-08T23:35:21.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.768+0000 7f48f2548700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 0x7f48d40074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.769+0000 7f48f2548700 1 -- 172.21.15.33:0/3416467891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48d400a5a0 con 0x7f48d4007020 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.769+0000 7f48f2548700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 0x7f48d40074b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004560 con 0x7f48d4007020 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4460 (secure 0 0 0) 0x7f48e801aa40 con 0x7f48d4007020 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 msgr2=0x7f48d40074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 0x7f48d40074b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e8011000 comp rx=0 tx=0).stop 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 msgr2=0x7f48d40074b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:21.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48e2ffd700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48d4007020 0x7f48d40074b0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000c00 tx=0x7f48e8011000 comp rx=0 tx=0).stop 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:21.769 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 18, 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:21.770 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:21.772 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.773 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.774 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:21.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:21.776 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.778 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:21.779 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:21.780 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48d006ad80 msgr2=0x7f48d006d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48f47ac700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48d006ad80 0x7f48d006d240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0aa5f0 tx=0x7f48e400b040 comp rx=0 tx=0).stop 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 msgr2=0x7f48ec0c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48f47ac700 1 --2- 172.21.15.33:0/3416467891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48ec0c85c0 0x7f48ec0c57a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00e840 tx=0x7f48dc00ec00 comp rx=0 tx=0).stop 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.770+0000 7f48f2d49700 1 -- 172.21.15.33:0/3416467891 reap_dead start 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.771+0000 7f48f2d49700 1 -- 172.21.15.33:0/3416467891 reap_dead start 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.771+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 shutdown_connections 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.771+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 >> 172.21.15.33:0/3416467891 conn(0x7f48ec0c89c0 msgr2=0x7f48ec0c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.771+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 shutdown_connections 2024-05-08T23:35:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:21.771+0000 7f48f47ac700 1 -- 172.21.15.33:0/3416467891 wait complete. 2024-05-08T23:35:21.807 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:21.808 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:35:21.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.167+0000 7f370325c700 1 Processor -- start 2024-05-08T23:35:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.167+0000 7f370325c700 1 -- start start 2024-05-08T23:35:22.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.168+0000 7f370325c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36fc0caad0 0x7f36fc0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.168+0000 7f370325c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc15b350 con 0x7f36fc154970 2024-05-08T23:35:22.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.168+0000 7f370325c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc15e8a0 con 0x7f36fc0cbbe0 2024-05-08T23:35:22.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.168+0000 7f370325c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc15a5d0 con 0x7f36fc0caad0 2024-05-08T23:35:22.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.169+0000 7f3700ff8700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f36fc154970 0x7f36fc06ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45892/0 (socket says 172.21.15.33:45892) 2024-05-08T23:35:22.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.169+0000 7f3700ff8700 1 -- 172.21.15.33:0/71243370 learned_addr learned my addr 172.21.15.33:0/71243370 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:35:22.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.170+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576290388 0 0) 0x7f36fc15b350 con 0x7f36fc154970 2024-05-08T23:35:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.170+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e40035b0 con 0x7f36fc154970 2024-05-08T23:35:22.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828624935 0 0) 0x7f36e40035b0 con 0x7f36fc154970 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36fc0caad0 msgr2=0x7f36fc0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36fc0caad0 0x7f36fc0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 >> v1:172.21.15.42:6789/0 conn(0x7f36fc0cbbe0 legacy=0x7f36fc15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc15e5c0 con 0x7f36fc154970 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f36f80019f0 con 0x7f36fc154970 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 >> v1:172.21.15.33:6789/0 conn(0x7f36fc154970 legacy=0x7f36fc06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4003a30 0x7f36e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4004850 0x7f36e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e4005210 0x7f36e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc15a5d0 con 0x7f36e4004850 2024-05-08T23:35:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e400c060 con 0x7f36e4005210 2024-05-08T23:35:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e400c220 con 0x7f36e4003a30 2024-05-08T23:35:22.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.172+0000 7f36f3fff700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4004850 0x7f36e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.173+0000 7f36f3fff700 1 -- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4003a30 msgr2=0x7f36e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.173+0000 7f36f3fff700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4003a30 0x7f36e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.173+0000 7f36f3fff700 1 -- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e4005210 msgr2=0x7f36e400ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.173+0000 7f36f3fff700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e4005210 0x7f36e400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.173+0000 7f36f3fff700 1 -- 172.21.15.33:0/71243370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc15e5c0 con 0x7f36e4004850 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.174+0000 7f36f3fff700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4004850 0x7f36e4004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0caf10 tx=0x7f36e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660321691725362 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.174+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec011620 con 0x7f36e4004850 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.174+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec011780 con 0x7f36e4004850 2024-05-08T23:35:22.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.174+0000 7f36f37fe700 1 -- 172.21.15.33:0/71243370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec011b30 con 0x7f36e4004850 2024-05-08T23:35:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.175+0000 7f370325c700 1 -- 172.21.15.33:0/71243370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4004850 msgr2=0x7f36e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.175+0000 7f370325c700 1 --2- 172.21.15.33:0/71243370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36e4004850 0x7f36e4004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0caf10 tx=0x7f36e4005880 comp rx=0 tx=0).stop 2024-05-08T23:35:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.176+0000 7f37017f9700 1 -- 172.21.15.33:0/71243370 reap_dead start 2024-05-08T23:35:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.176+0000 7f370325c700 1 -- 172.21.15.33:0/71243370 shutdown_connections 2024-05-08T23:35:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.176+0000 7f370325c700 1 -- 172.21.15.33:0/71243370 >> 172.21.15.33:0/71243370 conn(0x7f36fc06cd80 msgr2=0x7f36fc06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.202+0000 7f370325c700 1 -- 172.21.15.33:0/71243370 shutdown_connections 2024-05-08T23:35:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.203+0000 7f370325c700 1 -- 172.21.15.33:0/71243370 wait complete. 2024-05-08T23:35:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.205+0000 7f370325c700 1 Processor -- start 2024-05-08T23:35:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.205+0000 7f370325c700 1 -- start start 2024-05-08T23:35:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 0x7f36fc0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc0d05f0 0x7f36fc0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36fc0ce310 0x7f36fc200390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0c5a10 con 0x7f36fc0ce310 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc157b80 con 0x7f36fc0d05f0 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.207+0000 7f370325c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1fcf30 con 0x7f36e4004850 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.208+0000 7f3700ff8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 0x7f36fc0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.208+0000 7f3700ff8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 0x7f36fc0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:35826/0 (socket says 172.21.15.33:35826) 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.208+0000 7f3700ff8700 1 -- 172.21.15.33:0/68503269 learned_addr learned my addr 172.21.15.33:0/68503269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:22.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.208+0000 7f36f3fff700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc0d05f0 0x7f36fc0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.208+0000 7f37017f9700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36fc0ce310 0x7f36fc200390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.209+0000 7f3700ff8700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc0d05f0 msgr2=0x7f36fc0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.209+0000 7f3700ff8700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc0d05f0 0x7f36fc0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.209+0000 7f3700ff8700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36fc0ce310 msgr2=0x7f36fc200390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.209+0000 7f3700ff8700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36fc0ce310 0x7f36fc200390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.209+0000 7f3700ff8700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc1fddb0 con 0x7f36e4004850 2024-05-08T23:35:22.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.212+0000 7f3700ff8700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 0x7f36fc0c9370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36f800a060 tx=0x7f36f800a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.212+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80070d0 con 0x7f36e4004850 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.212+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8007230 con 0x7f36e4004850 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.212+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80075c0 con 0x7f36e4004850 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.212+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc0ce790 con 0x7f36e4004850 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.213+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc1fd1a0 con 0x7f36e4004850 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.214+0000 7f370325c700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 0x7f36fc2008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.214+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc074690 con 0x7f36fc1fd4e0 2024-05-08T23:35:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.217+0000 7f36f3fff700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 0x7f36fc2008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.217+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f36f8007720 con 0x7f36e4004850 2024-05-08T23:35:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.217+0000 7f36f17fa700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f36e806ad60 0x7f36e806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.217+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f36f80897b0 con 0x7f36e4004850 2024-05-08T23:35:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.218+0000 7f37017f9700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f36e806ad60 0x7f36e806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.219+0000 7f36f3fff700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36fc0cdc60 con 0x7f36fc1fd4e0 2024-05-08T23:35:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.219+0000 7f37017f9700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f36e806ad60 0x7f36e806d220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0cbfc0 tx=0x7f36f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.220+0000 7f36f3fff700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 0x7f36fc2008d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0caf10 tx=0x7f36ec004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec020040 con 0x7f36fc1fd4e0 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f36ec011e90 con 0x7f36fc1fd4e0 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 msgr2=0x7f36fc2008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 0x7f36fc2008d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0caf10 tx=0x7f36ec004860 comp rx=0 tx=0).stop 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 msgr2=0x7f36fc2008d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:22.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.222+0000 7f36f17fa700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd4e0 0x7f36fc2008d0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0caf10 tx=0x7f36ec004860 comp rx=0 tx=0).stop 2024-05-08T23:35:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.236+0000 7f370325c700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 0x7f36fc200e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.236+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0cdaf0 con 0x7f36fc1fd8c0 2024-05-08T23:35:22.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.237+0000 7f36f3fff700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 0x7f36fc200e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.238+0000 7f36f3fff700 1 -- 172.21.15.33:0/68503269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f36fc15ab50 con 0x7f36fc1fd8c0 2024-05-08T23:35:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.239+0000 7f36f3fff700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 0x7f36fc200e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0ca8d0 tx=0x7f36ec011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.239+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec02eb00 con 0x7f36fc1fd8c0 2024-05-08T23:35:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.241+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5111 (secure 0 0 0) 0x7f36ec02ec60 con 0x7f36fc1fd8c0 2024-05-08T23:35:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.241+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 msgr2=0x7f36fc200e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.241+0000 7f36f17fa700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 0x7f36fc200e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0ca8d0 tx=0x7f36ec011000 comp rx=0 tx=0).stop 2024-05-08T23:35:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.241+0000 7f36f17fa700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 msgr2=0x7f36fc200e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.241+0000 7f36f17fa700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36fc1fd8c0 0x7f36fc200e10 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0ca8d0 tx=0x7f36ec011000 comp rx=0 tx=0).stop 2024-05-08T23:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 18, 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f36e806ad60 msgr2=0x7f36e806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f36e806ad60 0x7f36e806d220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0cbfc0 tx=0x7f36f4009040 comp rx=0 tx=0).stop 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 msgr2=0x7f36fc0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 --2- 172.21.15.33:0/68503269 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f36e4004850 0x7f36fc0c9370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36f800a060 tx=0x7f36f800a370 comp rx=0 tx=0).stop 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f37017f9700 1 -- 172.21.15.33:0/68503269 reap_dead start 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f37017f9700 1 -- 172.21.15.33:0/68503269 reap_dead start 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 shutdown_connections 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.250+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 >> 172.21.15.33:0/68503269 conn(0x7f36fc06cd80 msgr2=0x7f36fc1537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.251+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 shutdown_connections 2024-05-08T23:35:22.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.251+0000 7f370325c700 1 -- 172.21.15.33:0/68503269 wait complete. 2024-05-08T23:35:22.289 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:22.289 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:35:22.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:35:22.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.629+0000 7f203ab7c700 1 Processor -- start 2024-05-08T23:35:22.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.630+0000 7f203ab7c700 1 -- start start 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 0x7f203406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f20340c68e0 0x7f2034071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f203406bfe0 0x7f2034071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2034157310 con 0x7f203406bfe0 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20341582e0 con 0x7f20340c68e0 2024-05-08T23:35:22.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.631+0000 7f203ab7c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2034157e50 con 0x7f20340c6310 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.636+0000 7f2033fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f20340c68e0 0x7f2034071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.636+0000 7f2038918700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 0x7f203406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.636+0000 7f2038918700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 0x7f203406baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:35840/0 (socket says 172.21.15.33:35840) 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.636+0000 7f2038918700 1 -- 172.21.15.33:0/2631361878 learned_addr learned my addr 172.21.15.33:0/2631361878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.637+0000 7f2039119700 1 --2- 172.21.15.33:0/2631361878 >> v2:172.21.15.33:3300/0 conn(0x7f203406bfe0 0x7f2034071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.637+0000 7f2039119700 1 -- 172.21.15.33:0/2631361878 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 msgr2=0x7f203406baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:22.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.638+0000 7f2039119700 1 --2- 172.21.15.33:0/2631361878 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 0x7f203406baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.638+0000 7f2039119700 1 -- 172.21.15.33:0/2631361878 >> v2:172.21.15.42:3300/0 conn(0x7f20340c68e0 msgr2=0x7f2034071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:22.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.638+0000 7f2039119700 1 --2- 172.21.15.33:0/2631361878 >> v2:172.21.15.42:3300/0 conn(0x7f20340c68e0 0x7f2034071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.638+0000 7f2039119700 1 -- 172.21.15.33:0/2631361878 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340c66f0 con 0x7f203406bfe0 2024-05-08T23:35:22.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.638+0000 7f2039119700 1 --2- 172.21.15.33:0/2631361878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f203406bfe0 0x7f2034071c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f203406fcb0 tx=0x7f202c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b27c90dd562f79d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.639+0000 7f20337fe700 1 -- 172.21.15.33:0/2631361878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c010070 con 0x7f203406bfe0 2024-05-08T23:35:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.639+0000 7f20337fe700 1 -- 172.21.15.33:0/2631361878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00b2e0 con 0x7f203406bfe0 2024-05-08T23:35:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.639+0000 7f20337fe700 1 -- 172.21.15.33:0/2631361878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00dc10 con 0x7f203406bfe0 2024-05-08T23:35:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.640+0000 7f203ab7c700 1 -- 172.21.15.33:0/2631361878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f203406bfe0 msgr2=0x7f2034071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.640+0000 7f203ab7c700 1 --2- 172.21.15.33:0/2631361878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f203406bfe0 0x7f2034071c50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f203406fcb0 tx=0x7f202c00d330 comp rx=0 tx=0).stop 2024-05-08T23:35:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 -- 172.21.15.33:0/2631361878 shutdown_connections 2024-05-08T23:35:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 --2- 172.21.15.33:0/2631361878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f203406bfe0 0x7f2034071c50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 --2- 172.21.15.33:0/2631361878 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f20340c6310 0x7f203406baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 --2- 172.21.15.33:0/2631361878 >> v2:172.21.15.42:3300/0 conn(0x7f20340c68e0 0x7f2034071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 -- 172.21.15.33:0/2631361878 >> 172.21.15.33:0/2631361878 conn(0x7f203406d2b0 msgr2=0x7f203406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:22.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 -- 172.21.15.33:0/2631361878 shutdown_connections 2024-05-08T23:35:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.641+0000 7f203ab7c700 1 -- 172.21.15.33:0/2631361878 wait complete. 2024-05-08T23:35:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.644+0000 7f203ab7c700 1 Processor -- start 2024-05-08T23:35:22.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.644+0000 7f203ab7c700 1 -- start start 2024-05-08T23:35:22.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f203406bfe0 0x7f20340c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 0x7f20341efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20340c68e0 0x7f20341f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340c9a40 con 0x7f20340c6310 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341fde60 con 0x7f20340c68e0 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f203ab7c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2034154d40 con 0x7f203406bfe0 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.646+0000 7f2033fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 0x7f20341efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 0x7f20341efb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60094/0 (socket says 172.21.15.33:60094) 2024-05-08T23:35:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 -- 172.21.15.33:0/796934942 learned_addr learned my addr 172.21.15.33:0/796934942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f203406bfe0 msgr2=0x7f20340c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f203406bfe0 0x7f20340c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20340c68e0 msgr2=0x7f20341f0090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20340c68e0 0x7f20341f0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.647+0000 7f2033fff700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c00bc80 con 0x7f20340c6310 2024-05-08T23:35:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.648+0000 7f2033fff700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 0x7f20341efb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f202800e9e0 tx=0x7f202800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.649+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028009710 con 0x7f20340c6310 2024-05-08T23:35:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.649+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202800c850 con 0x7f20340c6310 2024-05-08T23:35:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.649+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202800cbc0 con 0x7f20340c6310 2024-05-08T23:35:22.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.649+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20341fd8e0 con 0x7f20340c6310 2024-05-08T23:35:22.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.649+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2034154410 con 0x7f20340c6310 2024-05-08T23:35:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.655+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f202800cd20 con 0x7f20340c6310 2024-05-08T23:35:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.656+0000 7f20317fa700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f201806ad90 0x7f201806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.656+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f2028089740 con 0x7f20340c6310 2024-05-08T23:35:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.657+0000 7f203ab7c700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 0x7f2010004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.657+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2010007f20 con 0x7f2010002a40 2024-05-08T23:35:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.657+0000 7f2038918700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f201806ad90 0x7f201806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.658+0000 7f2039119700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 0x7f2010004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.658+0000 7f2038918700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f201806ad90 0x7f201806d250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2024006b20 tx=0x7f2024006ab0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.658+0000 7f2039119700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20100080e0 con 0x7f2010002a40 2024-05-08T23:35:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.658+0000 7f2039119700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 0x7f2010004f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f203406fcb0 tx=0x7f202c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004450 con 0x7f2010002a40 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f202c00b450 con 0x7f2010002a40 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 msgr2=0x7f2010004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 0x7f2010004f00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f203406fcb0 tx=0x7f202c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 msgr2=0x7f2010004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.659+0000 7f20317fa700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010002a40 0x7f2010004f00 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f203406fcb0 tx=0x7f202c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.693+0000 7f203ab7c700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 0x7f2010007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.693+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201000a3e0 con 0x7f2010006f70 2024-05-08T23:35:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.695+0000 7f2039119700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 0x7f2010007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:22.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.698+0000 7f2039119700 1 -- 172.21.15.33:0/796934942 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f201000a5a0 con 0x7f2010006f70 2024-05-08T23:35:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.700+0000 7f2039119700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 0x7f2010007400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20100080b0 tx=0x7f202c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.701+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0316a0 con 0x7f2010006f70 2024-05-08T23:35:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.702+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4968 (secure 0 0 0) 0x7f202c0282e0 con 0x7f2010006f70 2024-05-08T23:35:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.702+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 msgr2=0x7f2010007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.702+0000 7f20317fa700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 0x7f2010007400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20100080b0 tx=0x7f202c013000 comp rx=0 tx=0).stop 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.702+0000 7f20317fa700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 msgr2=0x7f2010007400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.702+0000 7f20317fa700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2010006f70 0x7f2010007400 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20100080b0 tx=0x7f202c013000 comp rx=0 tx=0).stop 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:35:22.701 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 28, 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 18, 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:22.702 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.703 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:22.704 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:22.705 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.706 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:22.709 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:22.710 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:22.712 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:35:22.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:22.714 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f201806ad90 msgr2=0x7f201806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f201806ad90 0x7f201806d250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2024006b20 tx=0x7f2024006ab0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 msgr2=0x7f20341efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 --2- 172.21.15.33:0/796934942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20340c6310 0x7f20341efb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f202800e9e0 tx=0x7f202800eda0 comp rx=0 tx=0).stop 2024-05-08T23:35:22.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f2039119700 1 -- 172.21.15.33:0/796934942 reap_dead start 2024-05-08T23:35:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 shutdown_connections 2024-05-08T23:35:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 >> 172.21.15.33:0/796934942 conn(0x7f203406d2b0 msgr2=0x7f2034071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.709+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 shutdown_connections 2024-05-08T23:35:22.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:22.712+0000 7f203ab7c700 1 -- 172.21.15.33:0/796934942 wait complete. 2024-05-08T23:35:22.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:35:22.737 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-05-08T23:35:22.737 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-05-08T23:35:22.737 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-08T23:35:22.737 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T23:35:22.737 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T23:35:22.737 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:35:22.760 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T23:35:22.760 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-05-08T23:35:22.760 INFO:tasks.ceph.mon.b:Sent signal 19 2024-05-08T23:35:22.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T23:35:37.761 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-05-08T23:35:37.762 INFO:tasks.ceph.mon.b:Sent signal 18 2024-05-08T23:35:37.762 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:35:37.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:35:38.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.315+0000 7f9bc2529700 1 Processor -- start 2024-05-08T23:35:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.315+0000 7f9bc2529700 1 -- start start 2024-05-08T23:35:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.317+0000 7f9bc2529700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.317+0000 7f9bc2529700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0c5dc0 con 0x7f9bbc0c7ae0 2024-05-08T23:35:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.317+0000 7f9bc2529700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc162c90 con 0x7f9bbc1556e0 2024-05-08T23:35:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.317+0000 7f9bc2529700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc06b180 con 0x7f9bbc0c9050 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:50522/0 (socket says 172.21.15.33:50522) 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 learned_addr learned my addr 172.21.15.33:0/962846998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.318+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.520+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:38.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.520+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:38.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.520+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:38.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.520+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:38.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.521+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.925+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.926+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.926+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.926+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:38.926+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:35:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:39.735+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:39.735+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:39.735+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:39.735+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:39.735+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:35:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9bbc06b610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9bbc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9bbc1556e0 legacy=0x7f9bbc070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:41.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.33:6789/0 conn(0x7f9bbc0c7ae0 legacy=0x7f9bbc169070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc06b180 con 0x7f9ba8006330 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8008ea0 con 0x7f9ba80037c0 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 msgr2=0x7f9ba8005c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.318+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.319+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.319+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:41.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.319+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.319+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:41.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.519+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 msgr2=0x7f9ba8005c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:41.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.519+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:41.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.520+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.521+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.521+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.521+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.521+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.920+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 msgr2=0x7f9ba8005c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.920+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:35:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.923+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.923+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.923+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.923+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:41.923+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:35:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.721+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 msgr2=0x7f9ba8005c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.722+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:35:42.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.724+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:42.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.725+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:42.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.725+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:42.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.725+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:42.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:42.725+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:35:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 msgr2=0x7f9ba8005c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80037c0 0x7f9ba8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.318+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8006330 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.318+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc1629b0 con 0x7f9bbc0c7ae0 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc06b180 con 0x7f9bbc0c9050 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8008ea0 con 0x7f9ba80092a0 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bc0ac6700 1 -- 172.21.15.33:0/962846998 reap_dead start 2024-05-08T23:35:44.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 msgr2=0x7f9ba800b6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:44.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.319+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:44.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.320+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.322+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:35:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.322+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.322+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.322+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.520+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 msgr2=0x7f9ba800b6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:44.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.520+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.524+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.525+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.525+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.525+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.525+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.922+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 msgr2=0x7f9ba800b6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.922+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:35:44.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.929+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.929+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.929+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.929+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:44.929+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:35:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.723+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 msgr2=0x7f9ba800b6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.723+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:35:45.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.731+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.732+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.732+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.732+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:45.732+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:35:47.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.320+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 msgr2=0x7f9ba800b6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:47.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.320+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba80092a0 0x7f9ba800b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.320+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 msgr2=0x7f9ba800dca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:47.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.320+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.42:3300/0 conn(0x7f9bbc0c9050 0x7f9ba800dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.33:6789/0 conn(0x7f9bbc0c7ae0 legacy=0x7f9ba80102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bb9ffb700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba8012b90 con 0x7f9ba8012120 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bb9ffb700 1 -- 172.21.15.33:0/962846998 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc06b180 con 0x7f9ba8011bc0 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:47.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.321+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.522+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.522+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:35:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.522+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 msgr2=0x7f9ba80087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:35:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.522+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:35:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.522+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:35:47.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.781+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9ba8012120 legacy=0x7f9ba8012d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T23:35:47.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.781+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9ba8012120 legacy=0x7f9ba8012d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:35:47.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.781+0000 7f9bbb7fe700 1 --1- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9ba8012120 0x7f9ba8012d40 :-1 s=OPENED pgs=108 cs=1 l=1).handle_message read tag failed 2024-05-08T23:35:47.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.781+0000 7f9bbb7fe700 1 --1- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9ba8012120 0x7f9ba8012d40 :-1 s=OPENED pgs=108 cs=1 l=1).fault on lossy channel, failing 2024-05-08T23:35:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.923+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> v2:172.21.15.33:3300/0 conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.923+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 >> v1:172.21.15.42:6789/0 conn(0x7f9ba8012120 legacy=0x7f9ba8012d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.923+0000 7f9bbbfff700 1 -- 172.21.15.33:0/962846998 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8012720 con 0x7f9ba8011bc0 2024-05-08T23:35:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.923+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8011bc0 0x7f9ba80087f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8008e70 tx=0x7f9bac005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c81a80f1696d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.924+0000 7f9bbaffd700 1 -- 172.21.15.33:0/962846998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac007550 con 0x7f9ba8011bc0 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.924+0000 7f9bbaffd700 1 -- 172.21.15.33:0/962846998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac0076b0 con 0x7f9ba8011bc0 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.924+0000 7f9bbaffd700 1 -- 172.21.15.33:0/962846998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac001870 con 0x7f9ba8011bc0 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.925+0000 7f9bc2529700 1 -- 172.21.15.33:0/962846998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8011bc0 msgr2=0x7f9ba80087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.925+0000 7f9bc2529700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8011bc0 0x7f9ba80087f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8008e70 tx=0x7f9bac005de0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.925+0000 7f9bc0ac6700 1 -- 172.21.15.33:0/962846998 reap_dead start 2024-05-08T23:35:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.927+0000 7f9bc2529700 1 -- 172.21.15.33:0/962846998 shutdown_connections 2024-05-08T23:35:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.927+0000 7f9bc2529700 1 --2- 172.21.15.33:0/962846998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ba8011bc0 0x7f9ba80087f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.927+0000 7f9bc2529700 1 -- 172.21.15.33:0/962846998 >> 172.21.15.33:0/962846998 conn(0x7f9bbc0c8360 msgr2=0x7f9bbc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.927+0000 7f9bc2529700 1 -- 172.21.15.33:0/962846998 shutdown_connections 2024-05-08T23:35:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.927+0000 7f9bc2529700 1 -- 172.21.15.33:0/962846998 wait complete. 2024-05-08T23:35:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.929+0000 7f9bc2529700 1 Processor -- start 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.931+0000 7f9bc2529700 1 -- start start 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba8011bc0 0x7f9bbc1617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bbc200830 0x7f9bbc200cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc200410 con 0x7f9bbc15ba70 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc2011f0 con 0x7f9bbc200830 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.932+0000 7f9bc2529700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15b770 con 0x7f9ba8011bc0 2024-05-08T23:35:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.933+0000 7f9bbb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.933+0000 7f9bbb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57546/0 (socket says 172.21.15.33:57546) 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.933+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/747029280 learned_addr learned my addr 172.21.15.33:0/747029280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.934+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba8011bc0 msgr2=0x7f9bbc1617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.934+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba8011bc0 0x7f9bbc1617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.934+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bbc200830 msgr2=0x7f9bbc200cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.934+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bbc200830 0x7f9bbc200cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:47.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.934+0000 7f9bbb7fe700 1 -- 172.21.15.33:0/747029280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac00f040 con 0x7f9bbc15ba70 2024-05-08T23:35:47.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.938+0000 7f9bbb7fe700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0017a30 tx=0x7f9bb0017df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.944+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0016d90 con 0x7f9bbc15ba70 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.944+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb001e040 con 0x7f9bbc15ba70 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.944+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00246c0 con 0x7f9bbc15ba70 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.948+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc160610 con 0x7f9bbc15ba70 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.948+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc160bd0 con 0x7f9bbc15ba70 2024-05-08T23:35:47.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.950+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc200680 con 0x7f9bbc15ba70 2024-05-08T23:35:47.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.951+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9bb0024820 con 0x7f9bbc15ba70 2024-05-08T23:35:47.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.952+0000 7f9bb8ff9700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 0x7f9ba806efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:47.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.952+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9bb008b570 con 0x7f9bbc15ba70 2024-05-08T23:35:47.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.952+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 0x7f9ba806efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:47.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.953+0000 7f9bbbfff700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 0x7f9ba806efa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bac018000 tx=0x7f9bac00a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:47.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:47.961+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9bb0056650 con 0x7f9bbc15ba70 2024-05-08T23:35:48.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:35:48.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:35:48.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.599+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9bbc075460 con 0x7f9bbc15ba70 2024-05-08T23:35:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.600+0000 7f9bb8ff9700 1 -- 172.21.15.33:0/747029280 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f9bb0059c70 con 0x7f9bbc15ba70 2024-05-08T23:35:48.606 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":32,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:35:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.611+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 msgr2=0x7f9ba806efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.611+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 0x7f9ba806efa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bac018000 tx=0x7f9bac00a000 comp rx=0 tx=0).stop 2024-05-08T23:35:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.611+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 msgr2=0x7f9bbc0ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.611+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0017a30 tx=0x7f9bb0017df0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 shutdown_connections 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9ba806cae0 0x7f9ba806efa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bbc200830 0x7f9bbc200cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9ba8011bc0 0x7f9bbc1617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 --2- 172.21.15.33:0/747029280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bbc15ba70 0x7f9bbc0ca3a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.612+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 >> 172.21.15.33:0/747029280 conn(0x7f9bbc0c8360 msgr2=0x7f9bbc0c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.613+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 shutdown_connections 2024-05-08T23:35:48.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.613+0000 7f9bc2529700 1 -- 172.21.15.33:0/747029280 wait complete. 2024-05-08T23:35:48.633 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:35:48.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-08T23:35:48.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:35:48.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.903+0000 7f902ab50700 1 Processor -- start 2024-05-08T23:35:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.905+0000 7f902ab50700 1 -- start start 2024-05-08T23:35:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.907+0000 7f902ab50700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90240c70d0 0x7f90240c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.907+0000 7f902ab50700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9024158ee0 con 0x7f9024161c70 2024-05-08T23:35:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.907+0000 7f902ab50700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902406c730 con 0x7f90240722c0 2024-05-08T23:35:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.907+0000 7f902ab50700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90240c2e60 con 0x7f90240c70d0 2024-05-08T23:35:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90288ec700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f90240722c0 0x7f90240c69c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:43426/0 (socket says 172.21.15.33:43426) 2024-05-08T23:35:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90288ec700 1 -- 172.21.15.33:0/740161976 learned_addr learned my addr 172.21.15.33:0/740161976 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:35:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090955755 0 0) 0x7f9024158ee0 con 0x7f9024161c70 2024-05-08T23:35:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c0035b0 con 0x7f9024161c70 2024-05-08T23:35:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154174636 0 0) 0x7f902406c730 con 0x7f90240722c0 2024-05-08T23:35:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.909+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9024158ee0 con 0x7f90240722c0 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682756855 0 0) 0x7f900c0035b0 con 0x7f9024161c70 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 >> v2:172.21.15.33:3300/0 conn(0x7f90240c70d0 msgr2=0x7f90240c94e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 --2- 172.21.15.33:0/740161976 >> v2:172.21.15.33:3300/0 conn(0x7f90240c70d0 0x7f90240c94e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 >> v1:172.21.15.42:6789/0 conn(0x7f90240722c0 legacy=0x7f90240c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902406ca10 con 0x7f9024161c70 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f9014001670 con 0x7f9024161c70 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 >> v1:172.21.15.33:6789/0 conn(0x7f9024161c70 legacy=0x7f90241620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c004bf0 0x7f900c004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0037f0 0x7f900c003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 0x7f900c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90240c2e60 con 0x7f900c0042d0 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00c230 con 0x7f900c004bf0 2024-05-08T23:35:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.910+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c00c3f0 con 0x7f900c0037f0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90288ec700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c004bf0 0x7f900c004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 0x7f900c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f9023fff700 1 -- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0037f0 msgr2=0x7f900c003c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f9023fff700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0037f0 0x7f900c003c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 -- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0037f0 msgr2=0x7f900c003c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0037f0 0x7f900c003c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 -- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c004bf0 msgr2=0x7f900c004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f900c004bf0 0x7f900c004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 -- 172.21.15.33:0/740161976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902406ca10 con 0x7f900c0042d0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.911+0000 7f90290ed700 1 -- 172.21.15.33:0/740161976 reap_dead start 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.912+0000 7f90290ed700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 0x7f900c00ba10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9014001eb0 tx=0x7f900c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ab4e710cad026 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.912+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014016070 con 0x7f900c0042d0 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.912+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901400caa0 con 0x7f900c0042d0 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.912+0000 7f90237fe700 1 -- 172.21.15.33:0/740161976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901400ce30 con 0x7f900c0042d0 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 -- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 msgr2=0x7f900c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 0x7f900c00ba10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9014001eb0 tx=0x7f900c005880 comp rx=0 tx=0).stop 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 -- 172.21.15.33:0/740161976 shutdown_connections 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 --2- 172.21.15.33:0/740161976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f900c0042d0 0x7f900c00ba10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 -- 172.21.15.33:0/740161976 >> 172.21.15.33:0/740161976 conn(0x7f90240c89c0 msgr2=0x7f90240c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 -- 172.21.15.33:0/740161976 shutdown_connections 2024-05-08T23:35:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.913+0000 7f902ab50700 1 -- 172.21.15.33:0/740161976 wait complete. 2024-05-08T23:35:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.915+0000 7f902ab50700 1 Processor -- start 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.916+0000 7f902ab50700 1 -- start start 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0042d0 0x7f90241492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9024157910 0x7f9024160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 0x7f90241583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024157480 con 0x7f90241567c0 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024156eb0 con 0x7f9024157910 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f902ab50700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9024149830 con 0x7f900c0042d0 2024-05-08T23:35:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f9023fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9024157910 0x7f9024160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f90288ec700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0042d0 msgr2=0x7f90241492f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f90288ec700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0042d0 0x7f90241492f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.917+0000 7f90290ed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 0x7f90241583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 0x7f90241583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57560/0 (socket says 172.21.15.33:57560) 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 learned_addr learned my addr 172.21.15.33:0/1280731615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0042d0 msgr2=0x7f90241492f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f900c0042d0 0x7f90241492f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9024157910 msgr2=0x7f9024160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9024157910 0x7f9024160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9024155b70 con 0x7f90241567c0 2024-05-08T23:35:48.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.918+0000 7f90290ed700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 0x7f90241583e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f902415c330 tx=0x7f9014001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:48.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.919+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014016040 con 0x7f90241567c0 2024-05-08T23:35:48.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.919+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901400ce60 con 0x7f90241567c0 2024-05-08T23:35:48.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.919+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014001c20 con 0x7f90241567c0 2024-05-08T23:35:48.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.920+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902415fc10 con 0x7f90241567c0 2024-05-08T23:35:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.922+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9014003680 con 0x7f90241567c0 2024-05-08T23:35:48.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.922+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90241601d0 con 0x7f90241567c0 2024-05-08T23:35:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.923+0000 7f90217fa700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f901006acc0 0x7f901006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.923+0000 7f902ab50700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 0x7f9024158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.923+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9024068bc0 con 0x7f9024157020 2024-05-08T23:35:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.923+0000 7f90288ec700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f901006acc0 0x7f901006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.923+0000 7f9023fff700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 0x7f9024158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.924+0000 7f9023fff700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f902406d2b0 con 0x7f9024157020 2024-05-08T23:35:48.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.924+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9014005510 con 0x7f90241567c0 2024-05-08T23:35:48.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.924+0000 7f90288ec700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f901006acc0 0x7f901006d180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f901c0013d0 tx=0x7f901c001360 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.924+0000 7f9023fff700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 0x7f9024158920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9024161c40 tx=0x7f901800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018003bb0 con 0x7f9024157020 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9018003d10 con 0x7f9024157020 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 msgr2=0x7f9024158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 0x7f9024158920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9024161c40 tx=0x7f901800da90 comp rx=0 tx=0).stop 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 msgr2=0x7f9024158920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:48.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.925+0000 7f90217fa700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9024157020 0x7f9024158920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:48.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.940+0000 7f902ab50700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 0x7f90241f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:48.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.940+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902406cfd0 con 0x7f90241e7f30 2024-05-08T23:35:48.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.940+0000 7f9023fff700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 0x7f90241f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.941+0000 7f9023fff700 1 -- 172.21.15.33:0/1280731615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f902406d590 con 0x7f90241e7f30 2024-05-08T23:35:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.941+0000 7f9023fff700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 0x7f90241f7210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9024157cf0 tx=0x7f9018005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9018020070 con 0x7f90241e7f30 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5101 (secure 0 0 0) 0x7f901801b8e0 con 0x7f90241e7f30 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 msgr2=0x7f90241f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 0x7f90241f7210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9024157cf0 tx=0x7f9018005a30 comp rx=0 tx=0).stop 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 msgr2=0x7f90241f7210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.942+0000 7f90217fa700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241e7f30 0x7f90241f7210 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9024157cf0 tx=0x7f9018005a30 comp rx=0 tx=0).stop 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 32, 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:35:48.942 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:48.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:48.944 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:48.945 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:48.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:48.949 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:48.953 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:48.954 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:48.955 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f901006acc0 msgr2=0x7f901006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f902ab50700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f901006acc0 0x7f901006d180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f901c0013d0 tx=0x7f901c001360 comp rx=0 tx=0).stop 2024-05-08T23:35:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 msgr2=0x7f90241583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f902ab50700 1 --2- 172.21.15.33:0/1280731615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90241567c0 0x7f90241583e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f902415c330 tx=0x7f9014001930 comp rx=0 tx=0).stop 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 reap_dead start 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.943+0000 7f90290ed700 1 -- 172.21.15.33:0/1280731615 reap_dead start 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.944+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 shutdown_connections 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.944+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 >> 172.21.15.33:0/1280731615 conn(0x7f90240c89c0 msgr2=0x7f90240c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.944+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 shutdown_connections 2024-05-08T23:35:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:48.944+0000 7f902ab50700 1 -- 172.21.15.33:0/1280731615 wait complete. 2024-05-08T23:35:48.973 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:35:48.973 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:35:48.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:35:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.224+0000 7f2183d75700 1 Processor -- start 2024-05-08T23:35:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.225+0000 7f2183d75700 1 -- start start 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2183d75700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f217c15b360 0x7f217c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2183d75700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 0x7f217c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2183d75700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0711c0 con 0x7f217c0b1ad0 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2183d75700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c06e320 con 0x7f217c15b360 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2183d75700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c071710 con 0x7f217c0c9220 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2181310700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 msgr2=0x7f217c0729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.226+0000 7f2181310700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 0x7f217c0729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f217c15b360 0x7f217c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f217c15b360 0x7f217c06a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:41580/0 (socket says 172.21.15.33:41580) 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 -- 172.21.15.33:0/1966861107 learned_addr learned my addr 172.21.15.33:0/1966861107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 -- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 msgr2=0x7f217c0729c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:35:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 --2- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 0x7f217c0729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 -- 172.21.15.33:0/1966861107 >> v1:172.21.15.42:6789/0 conn(0x7f217c0b1ad0 legacy=0x7f217c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:35:49.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 -- 172.21.15.33:0/1966861107 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c071430 con 0x7f217c15b360 2024-05-08T23:35:49.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.227+0000 7f2181b11700 1 --2- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c15b360 0x7f217c06a990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21780098f0 tx=0x7f2178009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852a6e7434cb9dab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:49.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.231+0000 7f2180b0f700 1 -- 172.21.15.33:0/1966861107 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178004230 con 0x7f217c15b360 2024-05-08T23:35:49.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.231+0000 7f2180b0f700 1 -- 172.21.15.33:0/1966861107 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178004390 con 0x7f217c15b360 2024-05-08T23:35:49.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.231+0000 7f2180b0f700 1 -- 172.21.15.33:0/1966861107 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21780036a0 con 0x7f217c15b360 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 -- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c15b360 msgr2=0x7f217c06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 --2- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c15b360 0x7f217c06a990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21780098f0 tx=0x7f2178009c00 comp rx=0 tx=0).stop 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 -- 172.21.15.33:0/1966861107 shutdown_connections 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 --2- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c15b360 0x7f217c06a990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 --2- 172.21.15.33:0/1966861107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0c9220 0x7f217c0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 -- 172.21.15.33:0/1966861107 >> 172.21.15.33:0/1966861107 conn(0x7f217c06d2b0 msgr2=0x7f217c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 -- 172.21.15.33:0/1966861107 shutdown_connections 2024-05-08T23:35:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.233+0000 7f2183d75700 1 -- 172.21.15.33:0/1966861107 wait complete. 2024-05-08T23:35:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.235+0000 7f2183d75700 1 Processor -- start 2024-05-08T23:35:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.235+0000 7f2183d75700 1 -- start start 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0b1ad0 0x7f217c1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0c9220 0x7f217c0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 0x7f217c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c152890 con 0x7f217c15b360 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b17f0 con 0x7f217c0c9220 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2183d75700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1f3fe0 con 0x7f217c0b1ad0 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2181b11700 1 -- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0b1ad0 msgr2=0x7f217c1f3aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3301/0 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2181b11700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0b1ad0 0x7f217c1f3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 0x7f217c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 0x7f217c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57586/0 (socket says 172.21.15.33:57586) 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 learned_addr learned my addr 172.21.15.33:0/1957563427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2181310700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0c9220 0x7f217c0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0b1ad0 msgr2=0x7f217c1f3aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f217c0b1ad0 0x7f217c1f3aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0c9220 msgr2=0x7f217c0cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0c9220 0x7f217c0cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.236+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2178009550 con 0x7f217c15b360 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f2182312700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 0x7f217c152ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f217400c780 tx=0x7f217400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174003e10 con 0x7f217c15b360 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740050f0 con 0x7f217c15b360 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740054a0 con 0x7f217c15b360 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c06b050 con 0x7f217c15b360 2024-05-08T23:35:49.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.237+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c0c3840 con 0x7f217c15b360 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.238+0000 7f2183d75700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 0x7f217c150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.238+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0749b0 con 0x7f217c153220 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.238+0000 7f2181b11700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 0x7f217c150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.239+0000 7f2181b11700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f217c06c970 con 0x7f217c153220 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.239+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2174005600 con 0x7f217c15b360 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.239+0000 7f2181b11700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 0x7f217c150e20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21780098f0 tx=0x7f2178004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.239+0000 7f21727fc700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f216006ac40 0x7f216006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f21740884c0 con 0x7f217c15b360 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f2181310700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f216006ac40 0x7f216006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217800b660 con 0x7f217c153220 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f217800b7c0 con 0x7f217c153220 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 msgr2=0x7f217c150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 0x7f217c150e20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f21780098f0 tx=0x7f2178004750 comp rx=0 tx=0).stop 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 msgr2=0x7f217c150e20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f21727fc700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c153220 0x7f217c150e20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.240+0000 7f2181310700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f216006ac40 0x7f216006d100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f216c009450 tx=0x7f216c0093e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.261+0000 7f2183d75700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 0x7f217c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:35:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.261+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0c36a0 con 0x7f217c151360 2024-05-08T23:35:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.262+0000 7f2181b11700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 0x7f217c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:35:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.262+0000 7f2181b11700 1 -- 172.21.15.33:0/1957563427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f217c075460 con 0x7f217c151360 2024-05-08T23:35:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f2181b11700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 0x7f217c0ae890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f21780092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:35:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178024b40 con 0x7f217c151360 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7f217802d070 con 0x7f217c151360 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 msgr2=0x7f217c0ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 0x7f217c0ae890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f21780092c0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 msgr2=0x7f217c0ae890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.263+0000 7f21727fc700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c151360 0x7f217c0ae890 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:49.263 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 32, 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: 1 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:35:49.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:35:49.267 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:49.270 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:35:49.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.272 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.273 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:35:49.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:35:49.276 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:35:49.276 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:35:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f216006ac40 msgr2=0x7f216006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f216006ac40 0x7f216006d100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f216c009450 tx=0x7f216c0093e0 comp rx=0 tx=0).stop 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 msgr2=0x7f217c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 --2- 172.21.15.33:0/1957563427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f217c15b360 0x7f217c152ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f217400c780 tx=0x7f217400ca90 comp rx=0 tx=0).stop 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 reap_dead start 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2182312700 1 -- 172.21.15.33:0/1957563427 reap_dead start 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 shutdown_connections 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 >> 172.21.15.33:0/1957563427 conn(0x7f217c06d2b0 msgr2=0x7f217c06d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 shutdown_connections 2024-05-08T23:35:49.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:35:49.264+0000 7f2183d75700 1 -- 172.21.15.33:0/1957563427 wait complete. 2024-05-08T23:35:49.283 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:35:49.283 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:36:09.284 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T23:36:09.284 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-08T23:36:09.285 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T23:36:09.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T23:36:09.309 INFO:tasks.ceph.mon.c:Started 2024-05-08T23:36:09.309 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-05-08T23:36:09.310 INFO:tasks.ceph.mon.b:Sent signal 19 2024-05-08T23:36:09.310 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T23:36:18.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:36:18.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:36:24.310 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-05-08T23:36:24.311 INFO:tasks.ceph.mon.b:Sent signal 18 2024-05-08T23:36:24.311 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:36:24.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.759+0000 7f6ca6ee2700 1 Processor -- start 2024-05-08T23:36:24.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.760+0000 7f6ca6ee2700 1 -- start start 2024-05-08T23:36:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.762+0000 7f6ca6ee2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6ca00c70c0 0x7f6ca00c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.762+0000 7f6ca6ee2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca0157c30 con 0x7f6ca01577a0 2024-05-08T23:36:24.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.762+0000 7f6ca6ee2700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca00c8110 con 0x7f6ca0150510 2024-05-08T23:36:24.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.762+0000 7f6ca6ee2700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca00c7e30 con 0x7f6ca00c70c0 2024-05-08T23:36:24.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.764+0000 7f6ca547f700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6ca01577a0 0x7f6ca0160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56896/0 (socket says 172.21.15.33:56896) 2024-05-08T23:36:24.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.764+0000 7f6ca547f700 1 -- 172.21.15.33:0/984900363 learned_addr learned my addr 172.21.15.33:0/984900363 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.764+0000 7f6c9f7fe700 1 -- 172.21.15.33:0/984900363 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143684568 0 0) 0x7f6ca0157c30 con 0x7f6ca01577a0 2024-05-08T23:36:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.764+0000 7f6c9f7fe700 1 -- 172.21.15.33:0/984900363 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c880035b0 con 0x7f6ca01577a0 2024-05-08T23:36:24.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.765+0000 7f6c9ffff700 1 --2- 172.21.15.33:0/984900363 >> v2:172.21.15.33:3300/0 conn(0x7f6ca00c70c0 0x7f6ca00c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.766+0000 7f6c9ffff700 1 -- 172.21.15.33:0/984900363 >> v1:172.21.15.42:6789/0 conn(0x7f6ca0150510 legacy=0x7f6ca006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.766+0000 7f6c9ffff700 1 -- 172.21.15.33:0/984900363 >> v1:172.21.15.33:6789/0 conn(0x7f6ca01577a0 legacy=0x7f6ca0160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.766+0000 7f6c9ffff700 1 -- 172.21.15.33:0/984900363 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00c6300 con 0x7f6ca00c70c0 2024-05-08T23:36:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.766+0000 7f6c9ffff700 1 --2- 172.21.15.33:0/984900363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca00c7780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009840 tx=0x7f6c94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fea791b95c37a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.769+0000 7f6c9f7fe700 1 -- 172.21.15.33:0/984900363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94004230 con 0x7f6ca00c70c0 2024-05-08T23:36:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.769+0000 7f6c9f7fe700 1 -- 172.21.15.33:0/984900363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c94004390 con 0x7f6ca00c70c0 2024-05-08T23:36:24.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.769+0000 7f6c9f7fe700 1 -- 172.21.15.33:0/984900363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940036a0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.770+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/984900363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 msgr2=0x7f6ca00c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:24.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.770+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/984900363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca00c7780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009840 tx=0x7f6c94009b50 comp rx=0 tx=0).stop 2024-05-08T23:36:24.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.775+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/984900363 shutdown_connections 2024-05-08T23:36:24.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.775+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/984900363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca00c7780 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:24.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.775+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/984900363 >> 172.21.15.33:0/984900363 conn(0x7f6ca00c8360 msgr2=0x7f6ca00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:24.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.809+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/984900363 shutdown_connections 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.809+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/984900363 wait complete. 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.811+0000 7f6ca6ee2700 1 Processor -- start 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.811+0000 7f6ca6ee2700 1 -- start start 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ca0150510 0x7f6ca014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ca01577a0 0x7f6ca01f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01e9ac0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01eddc0 con 0x7f6ca01577a0 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.812+0000 7f6ca6ee2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca01f2850 con 0x7f6ca0150510 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52966/0 (socket says 172.21.15.33:52966) 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 -- 172.21.15.33:0/2870312841 learned_addr learned my addr 172.21.15.33:0/2870312841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 -- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ca0150510 msgr2=0x7f6ca014d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ca0150510 0x7f6ca014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 -- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ca01577a0 msgr2=0x7f6ca01f2ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ca01577a0 0x7f6ca01f2ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.814+0000 7f6ca4c7e700 1 -- 172.21.15.33:0/2870312841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940094a0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.815+0000 7f6ca4c7e700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b470 tx=0x7f6c9000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.865+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011830 con 0x7f6ca00c70c0 2024-05-08T23:36:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.865+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c900119b0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.865+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011d60 con 0x7f6ca00c70c0 2024-05-08T23:36:24.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.865+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca01f2520 con 0x7f6ca00c70c0 2024-05-08T23:36:24.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.865+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca01e9cf0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.866+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca01f2380 con 0x7f6ca00c70c0 2024-05-08T23:36:24.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.878+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f6c9001a400 con 0x7f6ca00c70c0 2024-05-08T23:36:24.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.879+0000 7f6c9d7fa700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 0x7f6c8c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:24.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.879+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f6c90089900 con 0x7f6ca00c70c0 2024-05-08T23:36:24.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.879+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f6c900b69a0 con 0x7f6ca00c70c0 2024-05-08T23:36:24.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.880+0000 7f6c9ffff700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 0x7f6c8c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:24.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:24.881+0000 7f6c9ffff700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 0x7f6c8c06d220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009840 tx=0x7f6c94013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.755+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6ca01f2380 con 0x7f6ca00c70c0 2024-05-08T23:36:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.756+0000 7f6c9d7fa700 1 -- 172.21.15.33:0/2870312841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f6c90015610 con 0x7f6ca00c70c0 2024-05-08T23:36:25.754 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":34,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:25.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.760+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 msgr2=0x7f6c8c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:25.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.760+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 0x7f6c8c06d220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009840 tx=0x7f6c94013040 comp rx=0 tx=0).stop 2024-05-08T23:36:25.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.760+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 msgr2=0x7f6ca014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:25.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.760+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000b470 tx=0x7f6c9000b830 comp rx=0 tx=0).stop 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 shutdown_connections 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f6c8c06ad60 0x7f6c8c06d220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ca01577a0 0x7f6ca01f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ca00c70c0 0x7f6ca014cd90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 --2- 172.21.15.33:0/2870312841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f6ca0150510 0x7f6ca014d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.761+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 >> 172.21.15.33:0/2870312841 conn(0x7f6ca00c8360 msgr2=0x7f6ca014f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.763+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 shutdown_connections 2024-05-08T23:36:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:25.763+0000 7f6ca6ee2700 1 -- 172.21.15.33:0/2870312841 wait complete. 2024-05-08T23:36:28.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.208+0000 7ff086cdf700 1 Processor -- start 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.212+0000 7ff086cdf700 1 -- start start 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.213+0000 7ff086cdf700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff0800c70c0 0x7ff0800c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.213+0000 7ff086cdf700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.213+0000 7ff086cdf700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0801572f0 con 0x7ff080150510 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.213+0000 7ff086cdf700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff080157c30 con 0x7ff0800c70c0 2024-05-08T23:36:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.213+0000 7ff086cdf700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0800c8110 con 0x7ff0801577a0 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.216+0000 7ff07ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff0800c70c0 0x7ff0800c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.216+0000 7ff084a7b700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff080150510 0x7ff08006a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:46986/0 (socket says 172.21.15.33:46986) 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.216+0000 7ff084a7b700 1 -- 172.21.15.33:0/3497418728 learned_addr learned my addr 172.21.15.33:0/3497418728 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff08527c700 1 --2- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff08527c700 1 -- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 msgr2=0x7ff080156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff08527c700 1 -- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 msgr2=0x7ff080156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:36:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff08527c700 1 --2- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff08527c700 1 --2- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff07ffff700 1 -- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 msgr2=0x7ff080156db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff07ffff700 1 --2- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff07ffff700 1 -- 172.21.15.33:0/3497418728 >> v1:172.21.15.42:6789/0 conn(0x7ff080150510 legacy=0x7ff08006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.218+0000 7ff07ffff700 1 -- 172.21.15.33:0/3497418728 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800c7e30 con 0x7ff0800c70c0 2024-05-08T23:36:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.220+0000 7ff07ffff700 1 --2- 172.21.15.33:0/3497418728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0800c70c0 0x7ff0800c7780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff08006ffa0 tx=0x7ff074009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8c994f711a5e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.230+0000 7ff07f7fe700 1 -- 172.21.15.33:0/3497418728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074011830 con 0x7ff0800c70c0 2024-05-08T23:36:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.230+0000 7ff07f7fe700 1 -- 172.21.15.33:0/3497418728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff074011990 con 0x7ff0800c70c0 2024-05-08T23:36:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.230+0000 7ff07f7fe700 1 -- 172.21.15.33:0/3497418728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074011d20 con 0x7ff0800c70c0 2024-05-08T23:36:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.231+0000 7ff086cdf700 1 -- 172.21.15.33:0/3497418728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0800c70c0 msgr2=0x7ff0800c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.231+0000 7ff086cdf700 1 --2- 172.21.15.33:0/3497418728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0800c70c0 0x7ff0800c7780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff08006ffa0 tx=0x7ff074009b20 comp rx=0 tx=0).stop 2024-05-08T23:36:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.241+0000 7ff086cdf700 1 -- 172.21.15.33:0/3497418728 shutdown_connections 2024-05-08T23:36:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.241+0000 7ff086cdf700 1 --2- 172.21.15.33:0/3497418728 >> v2:172.21.15.42:3300/0 conn(0x7ff0801577a0 0x7ff080156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.241+0000 7ff086cdf700 1 --2- 172.21.15.33:0/3497418728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0800c70c0 0x7ff0800c7780 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff08006ffa0 tx=0x7ff074009b20 comp rx=0 tx=0).stop 2024-05-08T23:36:29.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.241+0000 7ff086cdf700 1 -- 172.21.15.33:0/3497418728 >> 172.21.15.33:0/3497418728 conn(0x7ff0800c8360 msgr2=0x7ff0800c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.242+0000 7ff086cdf700 1 -- 172.21.15.33:0/3497418728 shutdown_connections 2024-05-08T23:36:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.242+0000 7ff086cdf700 1 -- 172.21.15.33:0/3497418728 wait complete. 2024-05-08T23:36:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.244+0000 7ff086cdf700 1 Processor -- start 2024-05-08T23:36:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.249+0000 7ff086cdf700 1 -- start start 2024-05-08T23:36:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0800c5110 0x7ff0800c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08014db80 con 0x7ff0801577a0 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080067550 con 0x7ff080150510 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.250+0000 7ff086cdf700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800c5a30 con 0x7ff0800c5110 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33096/0 (socket says 172.21.15.33:33096) 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff07ffff700 1 -- 172.21.15.33:0/1142774886 learned_addr learned my addr 172.21.15.33:0/1142774886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff084a7b700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff084a7b700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 msgr2=0x7ff08014d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff084a7b700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 msgr2=0x7ff08014d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff084a7b700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.251+0000 7ff084a7b700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0800c5110 msgr2=0x7ff0800c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0800c5110 0x7ff0800c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 msgr2=0x7ff08014d640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 -- 172.21.15.33:0/1142774886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740094a0 con 0x7ff0801577a0 2024-05-08T23:36:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.252+0000 7ff07ffff700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff074013000 tx=0x7ff074004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.255+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074004450 con 0x7ff0801577a0 2024-05-08T23:36:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.255+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0740045b0 con 0x7ff0801577a0 2024-05-08T23:36:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.255+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740038c0 con 0x7ff0801577a0 2024-05-08T23:36:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.255+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0801fcc00 con 0x7ff0801577a0 2024-05-08T23:36:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.255+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0801fd1c0 con 0x7ff0801577a0 2024-05-08T23:36:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.264+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff074003a20 con 0x7ff0801577a0 2024-05-08T23:36:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.265+0000 7ff07d7fa700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 0x7ff06c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.265+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff07408a890 con 0x7ff0801577a0 2024-05-08T23:36:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.265+0000 7ff084a7b700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 0x7ff06c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.267+0000 7ff084a7b700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 0x7ff06c06d250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff078005c90 tx=0x7ff078009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:29.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.287+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff080075460 con 0x7ff0801577a0 2024-05-08T23:36:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:29.300+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff074055b30 con 0x7ff0801577a0 2024-05-08T23:36:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.197+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff080068dc0 con 0x7ff0801577a0 2024-05-08T23:36:30.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.201+0000 7ff07d7fa700 1 -- 172.21.15.33:0/1142774886 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7ff074059150 con 0x7ff0801577a0 2024-05-08T23:36:30.203 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":34,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.206+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 msgr2=0x7ff06c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.206+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 0x7ff06c06d250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff078005c90 tx=0x7ff078009040 comp rx=0 tx=0).stop 2024-05-08T23:36:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.206+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 msgr2=0x7ff0800715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.206+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff074013000 tx=0x7ff074004750 comp rx=0 tx=0).stop 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 shutdown_connections 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff06c06ad90 0x7ff06c06d250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff0800c5110 0x7ff0800c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff080150510 0x7ff08014d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 --2- 172.21.15.33:0/1142774886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0801577a0 0x7ff0800715b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:30.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.213+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 >> 172.21.15.33:0/1142774886 conn(0x7ff0800c8360 msgr2=0x7ff0800ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:30.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.234+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 shutdown_connections 2024-05-08T23:36:30.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:30.240+0000 7ff086cdf700 1 -- 172.21.15.33:0/1142774886 wait complete. 2024-05-08T23:36:33.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.719+0000 7f59c0696700 1 Processor -- start 2024-05-08T23:36:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.720+0000 7f59c0696700 1 -- start start 2024-05-08T23:36:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.722+0000 7f59c0696700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f59b80728c0 0x7f59b80c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.722+0000 7f59c0696700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 0x7f59b815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.722+0000 7f59c0696700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b80c9380 con 0x7f59b80c5580 2024-05-08T23:36:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.722+0000 7f59c0696700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b80c80a0 con 0x7f59b80728c0 2024-05-08T23:36:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.722+0000 7f59c0696700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b80c84c0 con 0x7f59b80c5100 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.723+0000 7f59bdc31700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 0x7f59b815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.723+0000 7f59bdc31700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 0x7f59b815a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:58404/0 (socket says 172.21.15.33:58404) 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.723+0000 7f59bdc31700 1 -- 172.21.15.33:0/2420222087 learned_addr learned my addr 172.21.15.33:0/2420222087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 --2- 172.21.15.33:0/2420222087 >> v2:172.21.15.33:3300/0 conn(0x7f59b80728c0 0x7f59b80c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 -- 172.21.15.33:0/2420222087 >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 msgr2=0x7f59b815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 --2- 172.21.15.33:0/2420222087 >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 0x7f59b815a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 -- 172.21.15.33:0/2420222087 >> v1:172.21.15.33:6789/0 conn(0x7f59b80c5580 legacy=0x7f59b815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 -- 172.21.15.33:0/2420222087 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b806c880 con 0x7f59b80728c0 2024-05-08T23:36:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.724+0000 7f59be432700 1 --2- 172.21.15.33:0/2420222087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80728c0 0x7f59b80c8e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59b80ca540 tx=0x7f59b400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5015e14a5cff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.728+0000 7f59bd430700 1 -- 172.21.15.33:0/2420222087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4018ad0 con 0x7f59b80728c0 2024-05-08T23:36:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.728+0000 7f59bd430700 1 -- 172.21.15.33:0/2420222087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4018c30 con 0x7f59b80728c0 2024-05-08T23:36:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.728+0000 7f59bd430700 1 -- 172.21.15.33:0/2420222087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b400d280 con 0x7f59b80728c0 2024-05-08T23:36:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.729+0000 7f59c0696700 1 -- 172.21.15.33:0/2420222087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80728c0 msgr2=0x7f59b80c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.729+0000 7f59c0696700 1 --2- 172.21.15.33:0/2420222087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80728c0 0x7f59b80c8e40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59b80ca540 tx=0x7f59b400df40 comp rx=0 tx=0).stop 2024-05-08T23:36:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.730+0000 7f59c0696700 1 -- 172.21.15.33:0/2420222087 shutdown_connections 2024-05-08T23:36:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.730+0000 7f59c0696700 1 --2- 172.21.15.33:0/2420222087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80728c0 0x7f59b80c8e40 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59b80ca540 tx=0x7f59b400df40 comp rx=0 tx=0).stop 2024-05-08T23:36:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.730+0000 7f59c0696700 1 --2- 172.21.15.33:0/2420222087 >> v2:172.21.15.42:3300/0 conn(0x7f59b80c5100 0x7f59b815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.730+0000 7f59c0696700 1 -- 172.21.15.33:0/2420222087 >> 172.21.15.33:0/2420222087 conn(0x7f59b80c9620 msgr2=0x7f59b80c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.738+0000 7f59c0696700 1 -- 172.21.15.33:0/2420222087 shutdown_connections 2024-05-08T23:36:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.738+0000 7f59c0696700 1 -- 172.21.15.33:0/2420222087 wait complete. 2024-05-08T23:36:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.740+0000 7f59c0696700 1 Processor -- start 2024-05-08T23:36:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.740+0000 7f59c0696700 1 -- start start 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59b80c5100 0x7f59b8153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f59b8153300 0x7f59b8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80cbb90 con 0x7f59b80c5580 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81ffe70 con 0x7f59b80c5100 2024-05-08T23:36:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.742+0000 7f59c0696700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81ecc40 con 0x7f59b8153300 2024-05-08T23:36:33.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.748+0000 7f59bdc31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:33.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.748+0000 7f59bdc31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33112/0 (socket says 172.21.15.33:33112) 2024-05-08T23:36:33.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.748+0000 7f59bdc31700 1 -- 172.21.15.33:0/479739496 learned_addr learned my addr 172.21.15.33:0/479739496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.748+0000 7f59bec33700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f59b8153300 0x7f59b8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 -- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f59b8153300 msgr2=0x7f59b8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f59b8153300 0x7f59b8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 -- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59b80c5100 msgr2=0x7f59b8153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59b80c5100 0x7f59b8153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 -- 172.21.15.33:0/479739496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b400d940 con 0x7f59b80c5580 2024-05-08T23:36:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.749+0000 7f59bdc31700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59a800daa0 tx=0x7f59a800ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.750+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800a970 con 0x7f59b80c5580 2024-05-08T23:36:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.750+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a800aad0 con 0x7f59b80c5580 2024-05-08T23:36:33.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.750+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800ae80 con 0x7f59b80c5580 2024-05-08T23:36:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.751+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b8151b70 con 0x7f59b80c5580 2024-05-08T23:36:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.751+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b8200380 con 0x7f59b80c5580 2024-05-08T23:36:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.752+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b8151e50 con 0x7f59b80c5580 2024-05-08T23:36:33.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.759+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f59a8010470 con 0x7f59b80c5580 2024-05-08T23:36:33.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.760+0000 7f59aeffd700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 0x7f599c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:33.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.760+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f59a8089d90 con 0x7f59b80c5580 2024-05-08T23:36:33.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.760+0000 7f59be432700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 0x7f599c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:33.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.762+0000 7f59be432700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 0x7f599c06d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59b80cdb20 tx=0x7f59b4009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:33.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:33.769+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f59a8054e80 con 0x7f59b80c5580 2024-05-08T23:36:34.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.522+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f59b8075460 con 0x7f59b80c5580 2024-05-08T23:36:34.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.524+0000 7f59aeffd700 1 -- 172.21.15.33:0/479739496 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f59a801e620 con 0x7f59b80c5580 2024-05-08T23:36:34.555 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 msgr2=0x7f599c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 0x7f599c06d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f59b80cdb20 tx=0x7f59b4009000 comp rx=0 tx=0).stop 2024-05-08T23:36:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 msgr2=0x7f59b8152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59a800daa0 tx=0x7f59a800ddb0 comp rx=0 tx=0).stop 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 shutdown_connections 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f599c06ac90 0x7f599c06d150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f59b8153300 0x7f59b8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59b80c5100 0x7f59b8153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 --2- 172.21.15.33:0/479739496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b80c5580 0x7f59b8152000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.587+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 >> 172.21.15.33:0/479739496 conn(0x7f59b80c9620 msgr2=0x7f59b80cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.588+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 shutdown_connections 2024-05-08T23:36:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:34.588+0000 7f59c0696700 1 -- 172.21.15.33:0/479739496 wait complete. 2024-05-08T23:36:34.629 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:36:34.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:36:34.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:36:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.211+0000 7ff4ec78a700 1 Processor -- start 2024-05-08T23:36:35.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.211+0000 7ff4ec78a700 1 -- start start 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.212+0000 7ff4ec78a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4e40c6310 0x7ff4e406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.213+0000 7ff4ec78a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e4161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.213+0000 7ff4ec78a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e41582e0 con 0x7ff4e40c68e0 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.213+0000 7ff4ec78a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e4157310 con 0x7ff4e40c6310 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.213+0000 7ff4ec78a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e4157e50 con 0x7ff4e4071e20 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4e40c6310 0x7ff4e406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff4e40c6310 0x7ff4e406baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33118/0 (socket says 172.21.15.33:33118) 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 -- 172.21.15.33:0/4062235738 learned_addr learned my addr 172.21.15.33:0/4062235738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 -- 172.21.15.33:0/4062235738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 msgr2=0x7ff4e4161bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 --2- 172.21.15.33:0/4062235738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e4161bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 -- 172.21.15.33:0/4062235738 >> v1:172.21.15.42:6789/0 conn(0x7ff4e40c68e0 legacy=0x7ff4e4071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:35.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.214+0000 7ff4ea526700 1 -- 172.21.15.33:0/4062235738 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e40c66f0 con 0x7ff4e40c6310 2024-05-08T23:36:35.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.215+0000 7ff4ea526700 1 --2- 172.21.15.33:0/4062235738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e406baa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098f0 tx=0x7ff4dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c1d04481c0f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.216+0000 7ff4e9524700 1 -- 172.21.15.33:0/4062235738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004230 con 0x7ff4e40c6310 2024-05-08T23:36:35.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.216+0000 7ff4e9524700 1 -- 172.21.15.33:0/4062235738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc004390 con 0x7ff4e40c6310 2024-05-08T23:36:35.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.216+0000 7ff4e9524700 1 -- 172.21.15.33:0/4062235738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0036a0 con 0x7ff4e40c6310 2024-05-08T23:36:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.217+0000 7ff4ec78a700 1 -- 172.21.15.33:0/4062235738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 msgr2=0x7ff4e406baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.217+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/4062235738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e406baa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc0098f0 tx=0x7ff4dc009c00 comp rx=0 tx=0).stop 2024-05-08T23:36:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 -- 172.21.15.33:0/4062235738 shutdown_connections 2024-05-08T23:36:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/4062235738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e4161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/4062235738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e406baa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 -- 172.21.15.33:0/4062235738 >> 172.21.15.33:0/4062235738 conn(0x7ff4e406d2b0 msgr2=0x7ff4e406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 -- 172.21.15.33:0/4062235738 shutdown_connections 2024-05-08T23:36:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.226+0000 7ff4ec78a700 1 -- 172.21.15.33:0/4062235738 wait complete. 2024-05-08T23:36:35.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.228+0000 7ff4ec78a700 1 Processor -- start 2024-05-08T23:36:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.229+0000 7ff4ec78a700 1 -- start start 2024-05-08T23:36:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e41fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e40c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e40c68e0 0x7ff4e406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e41f0230 con 0x7ff4e40c6310 2024-05-08T23:36:35.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e41f0740 con 0x7ff4e40c68e0 2024-05-08T23:36:35.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.230+0000 7ff4ec78a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40c0840 con 0x7ff4e4071e20 2024-05-08T23:36:35.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.232+0000 7ff4ea526700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e41fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.232+0000 7ff4ea526700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e41fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:52064/0 (socket says 172.21.15.33:52064) 2024-05-08T23:36:35.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.232+0000 7ff4ea526700 1 -- 172.21.15.33:0/712986136 learned_addr learned my addr 172.21.15.33:0/712986136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:35.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.232+0000 7ff4e9d25700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e40c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 msgr2=0x7ff4e41fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4e4071e20 0x7ff4e41fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e40c68e0 msgr2=0x7ff4e406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e40c68e0 0x7ff4e406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc009550 con 0x7ff4e40c6310 2024-05-08T23:36:35.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4e9d25700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e40c3520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400b410 tx=0x7ff4d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.233+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4011830 con 0x7ff4e40c6310 2024-05-08T23:36:35.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.234+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d40119b0 con 0x7ff4e40c6310 2024-05-08T23:36:35.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.234+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4011d60 con 0x7ff4e40c6310 2024-05-08T23:36:35.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.234+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e41f0460 con 0x7ff4e40c6310 2024-05-08T23:36:35.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.235+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e406ada0 con 0x7ff4e40c6310 2024-05-08T23:36:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.241+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff4d401a410 con 0x7ff4e40c6310 2024-05-08T23:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.242+0000 7ff4daffd700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff4c806ad90 0x7ff4c806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.242+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff4d4088db0 con 0x7ff4e40c6310 2024-05-08T23:36:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.242+0000 7ff4ea526700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff4c806ad90 0x7ff4c806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.243+0000 7ff4ea526700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff4c806ad90 0x7ff4c806d250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc004720 tx=0x7ff4dc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.246+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 0x7ff4e41fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.246+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40749b0 con 0x7ff4e406afc0 2024-05-08T23:36:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.246+0000 7ff4ead27700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 0x7ff4e41fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.248+0000 7ff4ead27700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4e4155180 con 0x7ff4e406afc0 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.249+0000 7ff4ead27700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 0x7ff4e41fd140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.249+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000cbc0 con 0x7ff4e406afc0 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.250+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff4e000cd20 con 0x7ff4e406afc0 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.250+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 msgr2=0x7ff4e41fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.250+0000 7ff4daffd700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 0x7ff4e41fd140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e000bce0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.250+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 msgr2=0x7ff4e41fd140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.250+0000 7ff4daffd700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e406afc0 0x7ff4e41fd140 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e000bce0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.273+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 0x7ff4e41fda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.273+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e4072a80 con 0x7ff4e41fd680 2024-05-08T23:36:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.274+0000 7ff4ead27700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 0x7ff4e41fda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.275+0000 7ff4ead27700 1 -- 172.21.15.33:0/712986136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff4e4157800 con 0x7ff4e41fd680 2024-05-08T23:36:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.275+0000 7ff4ead27700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 0x7ff4e41fda90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e0004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e001cae0 con 0x7ff4e41fd680 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5112 (secure 0 0 0) 0x7ff4e000ce90 con 0x7ff4e41fd680 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 msgr2=0x7ff4e41fda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 0x7ff4e41fda90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e0004c20 comp rx=0 tx=0).stop 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 msgr2=0x7ff4e41fda90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:35.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.276+0000 7ff4daffd700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e41fd680 0x7ff4e41fda90 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000b920 tx=0x7ff4e0004c20 comp rx=0 tx=0).stop 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:35.287 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 4, 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.288 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.289 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:35.290 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.294 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.297 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:36:35.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:35.300 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:35.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.306+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff4c806ad90 msgr2=0x7ff4c806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.306+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff4c806ad90 0x7ff4c806d250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc004720 tx=0x7ff4dc008030 comp rx=0 tx=0).stop 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.306+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 msgr2=0x7ff4e40c3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.306+0000 7ff4ec78a700 1 --2- 172.21.15.33:0/712986136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4e40c6310 0x7ff4e40c3520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400b410 tx=0x7ff4d4009f90 comp rx=0 tx=0).stop 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.306+0000 7ff4ead27700 1 -- 172.21.15.33:0/712986136 reap_dead start 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.308+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 shutdown_connections 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.308+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 >> 172.21.15.33:0/712986136 conn(0x7ff4e406d2b0 msgr2=0x7ff4e40c9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.308+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 shutdown_connections 2024-05-08T23:36:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.308+0000 7ff4ec78a700 1 -- 172.21.15.33:0/712986136 wait complete. 2024-05-08T23:36:35.348 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:35.348 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:36:35.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.843+0000 7f2e4307e700 1 Processor -- start 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.843+0000 7f2e4307e700 1 -- start start 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.844+0000 7f2e4307e700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2e3c0c2770 0x7f2e3c0c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.844+0000 7f2e4307e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 0x7f2e3c070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.844+0000 7f2e4307e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c06ba40 con 0x7f2e3c1589d0 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.844+0000 7f2e4307e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c15f180 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.844+0000 7f2e4307e700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0c8210 con 0x7f2e3c0c2770 2024-05-08T23:36:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.847+0000 7f2e40e1a700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2e3c1589d0 0x7f2e3c06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36302/0 (socket says 172.21.15.33:36302) 2024-05-08T23:36:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.847+0000 7f2e40e1a700 1 -- 172.21.15.33:0/772382336 learned_addr learned my addr 172.21.15.33:0/772382336 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:35.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.853+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/772382336 >> v2:172.21.15.42:3300/0 conn(0x7f2e3c0c2770 0x7f2e3c0c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.854+0000 7f2e3bfff700 1 -- 172.21.15.33:0/772382336 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 msgr2=0x7f2e3c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.847+0000 7f2e4161b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 0x7f2e3c070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.854+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/772382336 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 0x7f2e3c070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.854+0000 7f2e3bfff700 1 -- 172.21.15.33:0/772382336 >> v1:172.21.15.33:6789/0 conn(0x7f2e3c1589d0 legacy=0x7f2e3c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.854+0000 7f2e3bfff700 1 -- 172.21.15.33:0/772382336 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c070930 con 0x7f2e3c0c2770 2024-05-08T23:36:35.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.855+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/772382336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c0c2770 0x7f2e3c0c21a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69394c3e3c957819 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.855+0000 7f2e4161b700 1 --2- 172.21.15.33:0/772382336 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 0x7f2e3c070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:36:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.860+0000 7f2e3b7fe700 1 -- 172.21.15.33:0/772382336 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30011830 con 0x7f2e3c0c2770 2024-05-08T23:36:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.861+0000 7f2e3b7fe700 1 -- 172.21.15.33:0/772382336 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30011990 con 0x7f2e3c0c2770 2024-05-08T23:36:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.861+0000 7f2e3b7fe700 1 -- 172.21.15.33:0/772382336 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30011d00 con 0x7f2e3c0c2770 2024-05-08T23:36:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 -- 172.21.15.33:0/772382336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c0c2770 msgr2=0x7f2e3c0c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 --2- 172.21.15.33:0/772382336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c0c2770 0x7f2e3c0c21a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30009b20 comp rx=0 tx=0).stop 2024-05-08T23:36:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 -- 172.21.15.33:0/772382336 shutdown_connections 2024-05-08T23:36:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 --2- 172.21.15.33:0/772382336 >> v2:172.21.15.33:3300/0 conn(0x7f2e3c0c7d80 0x7f2e3c070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 --2- 172.21.15.33:0/772382336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c0c2770 0x7f2e3c0c21a0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30009b20 comp rx=0 tx=0).stop 2024-05-08T23:36:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 -- 172.21.15.33:0/772382336 >> 172.21.15.33:0/772382336 conn(0x7f2e3c0c8360 msgr2=0x7f2e3c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 -- 172.21.15.33:0/772382336 shutdown_connections 2024-05-08T23:36:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.862+0000 7f2e4307e700 1 -- 172.21.15.33:0/772382336 wait complete. 2024-05-08T23:36:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.864+0000 7f2e4307e700 1 Processor -- start 2024-05-08T23:36:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.865+0000 7f2e4307e700 1 -- start start 2024-05-08T23:36:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 0x7f2e3c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c1589d0 0x7f2e3c13feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c1403f0 0x7f2e3c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c13da10 con 0x7f2e3c1589d0 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c1595e0 con 0x7f2e3c1403f0 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.866+0000 7f2e4307e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c15a160 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.868+0000 7f2e40e1a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 0x7f2e3c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.868+0000 7f2e40e1a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 0x7f2e3c13e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:35560/0 (socket says 172.21.15.33:35560) 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.868+0000 7f2e40e1a700 1 -- 172.21.15.33:0/1206445917 learned_addr learned my addr 172.21.15.33:0/1206445917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.868+0000 7f2e4161b700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c1403f0 0x7f2e3c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.868+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c1589d0 0x7f2e3c13feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.869+0000 7f2e40e1a700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c1403f0 msgr2=0x7f2e3c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.869+0000 7f2e40e1a700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e3c1403f0 0x7f2e3c0c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.869+0000 7f2e40e1a700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c1589d0 msgr2=0x7f2e3c13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.869+0000 7f2e40e1a700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e3c1589d0 0x7f2e3c13feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.869+0000 7f2e40e1a700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300094a0 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.873+0000 7f2e40e1a700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 0x7f2e3c13e5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c490 tx=0x7f2e2c00c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.879+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c005d70 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.879+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0070d0 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.879+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c007460 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.879+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0c7510 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.879+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c159830 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.882+0000 7f2e4307e700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 0x7f2e1c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.882+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c007f20 con 0x7f2e1c002a40 2024-05-08T23:36:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.884+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2e2c0075c0 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.885+0000 7f2e397fa700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e2806ad60 0x7f2e2806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.885+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f2e2c0888d0 con 0x7f2e3c0c7d80 2024-05-08T23:36:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.886+0000 7f2e4161b700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e2806ad60 0x7f2e2806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.886+0000 7f2e4161b700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e2806ad60 0x7f2e2806d220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c140820 tx=0x7f2e34007540 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.890+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 0x7f2e1c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.894+0000 7f2e3bfff700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e1c0080e0 con 0x7f2e1c002a40 2024-05-08T23:36:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.899+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 0x7f2e1c004f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.902+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30020070 con 0x7f2e1c002a40 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.903+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2e300045e0 con 0x7f2e1c002a40 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.903+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 msgr2=0x7f2e1c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.903+0000 7f2e397fa700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 0x7f2e1c004f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30004750 comp rx=0 tx=0).stop 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.903+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 msgr2=0x7f2e1c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.903+0000 7f2e397fa700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c002a40 0x7f2e1c004f00 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c15c380 tx=0x7f2e30004750 comp rx=0 tx=0).stop 2024-05-08T23:36:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.921+0000 7f2e4307e700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 0x7f2e1c0072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.921+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c00a480 con 0x7f2e1c006e80 2024-05-08T23:36:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.923+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 0x7f2e1c0072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.923+0000 7f2e3bfff700 1 -- 172.21.15.33:0/1206445917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2e1c007830 con 0x7f2e1c006e80 2024-05-08T23:36:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.924+0000 7f2e3bfff700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 0x7f2e1c0072f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e30000c00 tx=0x7f2e30011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30003ab0 con 0x7f2e1c006e80 2024-05-08T23:36:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4967 (secure 0 0 0) 0x7f2e30003c10 con 0x7f2e1c006e80 2024-05-08T23:36:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 msgr2=0x7f2e1c0072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 0x7f2e1c0072f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e30000c00 tx=0x7f2e30011000 comp rx=0 tx=0).stop 2024-05-08T23:36:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 msgr2=0x7f2e1c0072f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.930+0000 7f2e397fa700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e1c006e80 0x7f2e1c0072f0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e30000c00 tx=0x7f2e30011000 comp rx=0 tx=0).stop 2024-05-08T23:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 5, 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e2806ad60 msgr2=0x7f2e2806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4307e700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e2806ad60 0x7f2e2806d220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c140820 tx=0x7f2e34007540 comp rx=0 tx=0).stop 2024-05-08T23:36:35.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 msgr2=0x7f2e3c13e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:35.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4307e700 1 --2- 172.21.15.33:0/1206445917 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e3c0c7d80 0x7f2e3c13e5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c490 tx=0x7f2e2c00c7a0 comp rx=0 tx=0).stop 2024-05-08T23:36:35.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4161b700 1 -- 172.21.15.33:0/1206445917 reap_dead start 2024-05-08T23:36:35.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.943+0000 7f2e4161b700 1 -- 172.21.15.33:0/1206445917 reap_dead start 2024-05-08T23:36:35.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.958+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 shutdown_connections 2024-05-08T23:36:35.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.958+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 >> 172.21.15.33:0/1206445917 conn(0x7f2e3c0c8360 msgr2=0x7f2e3c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:35.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.959+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 shutdown_connections 2024-05-08T23:36:35.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:35.959+0000 7f2e4307e700 1 -- 172.21.15.33:0/1206445917 wait complete. 2024-05-08T23:36:36.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:36.005 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:36:36.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:36:36.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.559+0000 7f0e8b23a700 1 Processor -- start 2024-05-08T23:36:36.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.559+0000 7f0e8b23a700 1 -- start start 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.560+0000 7f0e8b23a700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0e840c3a90 0x7f0e84158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.560+0000 7f0e8b23a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840c81b0 con 0x7f0e840c7dd0 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.560+0000 7f0e8b23a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840c2d60 con 0x7f0e840c8e50 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.561+0000 7f0e8b23a700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e84072ed0 con 0x7f0e840c3a90 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.561+0000 7f0e88fd6700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0e840c7dd0 0x7f0e8406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36310/0 (socket says 172.21.15.33:36310) 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.561+0000 7f0e88fd6700 1 -- 172.21.15.33:0/1176706073 learned_addr learned my addr 172.21.15.33:0/1176706073 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.561+0000 7f0e83fff700 1 --2- 172.21.15.33:0/1176706073 >> v2:172.21.15.42:3300/0 conn(0x7f0e840c3a90 0x7f0e84158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113390353 0 0) 0x7f0e840c81b0 con 0x7f0e840c7dd0 2024-05-08T23:36:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e6c0035b0 con 0x7f0e840c7dd0 2024-05-08T23:36:36.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126688275 0 0) 0x7f0e6c0035b0 con 0x7f0e840c7dd0 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 >> v2:172.21.15.42:3300/0 conn(0x7f0e840c3a90 msgr2=0x7f0e84158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 --2- 172.21.15.33:0/1176706073 >> v2:172.21.15.42:3300/0 conn(0x7f0e840c3a90 0x7f0e84158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 >> v1:172.21.15.42:6789/0 conn(0x7f0e840c8e50 legacy=0x7f0e840c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.562+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84158d30 con 0x7f0e840c7dd0 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f0e7c0019f0 con 0x7f0e840c7dd0 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 >> v1:172.21.15.33:6789/0 conn(0x7f0e840c7dd0 legacy=0x7f0e8406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e6c003a00 0x7f0e6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e6c004850 0x7f0e6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 0x7f0e6c0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84072ed0 con 0x7f0e6c009a00 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c014960 con 0x7f0e6c004850 2024-05-08T23:36:36.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c014b20 con 0x7f0e6c003a00 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.563+0000 7f0e897d7700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 0x7f0e6c0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 -- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e6c003a00 msgr2=0x7f0e6c003e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e6c003a00 0x7f0e6c003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 -- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e6c004850 msgr2=0x7f0e6c004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e6c004850 0x7f0e6c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 -- 172.21.15.33:0/1176706073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84158d30 con 0x7f0e6c009a00 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 -- 172.21.15.33:0/1176706073 reap_dead start 2024-05-08T23:36:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.564+0000 7f0e897d7700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 0x7f0e6c0052c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e78003f00 tx=0x7f0e6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c69d786e019a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:36.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.574+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78017a70 con 0x7f0e6c009a00 2024-05-08T23:36:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.574+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78017bd0 con 0x7f0e6c009a00 2024-05-08T23:36:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.574+0000 7f0e837fe700 1 -- 172.21.15.33:0/1176706073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78018020 con 0x7f0e6c009a00 2024-05-08T23:36:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.575+0000 7f0e8b23a700 1 -- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 msgr2=0x7f0e6c0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.575+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 0x7f0e6c0052c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e78003f00 tx=0x7f0e6c005880 comp rx=0 tx=0).stop 2024-05-08T23:36:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.577+0000 7f0e8b23a700 1 -- 172.21.15.33:0/1176706073 shutdown_connections 2024-05-08T23:36:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.577+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/1176706073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e6c009a00 0x7f0e6c0052c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.577+0000 7f0e8b23a700 1 -- 172.21.15.33:0/1176706073 >> 172.21.15.33:0/1176706073 conn(0x7f0e840c8360 msgr2=0x7f0e840c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.578+0000 7f0e8b23a700 1 -- 172.21.15.33:0/1176706073 shutdown_connections 2024-05-08T23:36:36.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.578+0000 7f0e8b23a700 1 -- 172.21.15.33:0/1176706073 wait complete. 2024-05-08T23:36:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.580+0000 7f0e8b23a700 1 Processor -- start 2024-05-08T23:36:36.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.580+0000 7f0e8b23a700 1 -- start start 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e8414ccb0 0x7f0e841541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8414d090 0x7f0e84154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 0x7f0e841eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8414e030 con 0x7f0e8414ccb0 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8414c760 con 0x7f0e8414d090 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.581+0000 7f0e8b23a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841539a0 con 0x7f0e84149ec0 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.582+0000 7f0e897d7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 0x7f0e841eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.582+0000 7f0e897d7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 0x7f0e841eea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:35584/0 (socket says 172.21.15.33:35584) 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.582+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 learned_addr learned my addr 172.21.15.33:0/3998537397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.582+0000 7f0e88fd6700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e8414ccb0 0x7f0e841541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.583+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8414d090 msgr2=0x7f0e84154710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:36:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.583+0000 7f0e897d7700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8414d090 0x7f0e84154710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.583+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e8414ccb0 msgr2=0x7f0e841541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.583+0000 7f0e897d7700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e8414ccb0 0x7f0e841541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.583+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8414dd50 con 0x7f0e84149ec0 2024-05-08T23:36:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.586+0000 7f0e897d7700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 0x7f0e841eea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e8406f9a0 tx=0x7f0e78003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:36.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.595+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e780185e0 con 0x7f0e84149ec0 2024-05-08T23:36:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.595+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7802e470 con 0x7f0e84149ec0 2024-05-08T23:36:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.595+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8414b110 con 0x7f0e84149ec0 2024-05-08T23:36:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.595+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8414b3f0 con 0x7f0e84149ec0 2024-05-08T23:36:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.596+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78018760 con 0x7f0e84149ec0 2024-05-08T23:36:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.597+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 0x7f0e841eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.597+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84074a20 con 0x7f0e8414bff0 2024-05-08T23:36:36.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.599+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e7802e820 con 0x7f0e84149ec0 2024-05-08T23:36:36.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.600+0000 7f0e817fa700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e7006ad60 0x7f0e7006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.600+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0e7809ad90 con 0x7f0e84149ec0 2024-05-08T23:36:36.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.600+0000 7f0e83fff700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e7006ad60 0x7f0e7006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.601+0000 7f0e83fff700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e7006ad60 0x7f0e7006d220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e840701d0 tx=0x7f0e74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:36.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.602+0000 7f0e88fd6700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 0x7f0e841eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.603+0000 7f0e88fd6700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e840c3290 con 0x7f0e8414bff0 2024-05-08T23:36:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.604+0000 7f0e88fd6700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 0x7f0e841eef40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c013420 con 0x7f0e8414bff0 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0e7c013580 con 0x7f0e8414bff0 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 msgr2=0x7f0e841eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 0x7f0e841eef40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c00e720 comp rx=0 tx=0).stop 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 msgr2=0x7f0e841eef40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:36.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.609+0000 7f0e817fa700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e8414bff0 0x7f0e841eef40 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c00e720 comp rx=0 tx=0).stop 2024-05-08T23:36:36.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.633+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 0x7f0e84148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:36.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.633+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8414b2a0 con 0x7f0e841ef480 2024-05-08T23:36:36.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.636+0000 7f0e88fd6700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 0x7f0e84148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:36.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.636+0000 7f0e88fd6700 1 -- 172.21.15.33:0/3998537397 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e840c2fb0 con 0x7f0e841ef480 2024-05-08T23:36:36.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.645+0000 7f0e88fd6700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 0x7f0e84148e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c001340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 6, 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c013ca0 con 0x7f0e841ef480 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4457 (secure 0 0 0) 0x7f0e7c013e00 con 0x7f0e841ef480 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 msgr2=0x7f0e84148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 0x7f0e84148e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c001340 comp rx=0 tx=0).stop 2024-05-08T23:36:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 msgr2=0x7f0e84148e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.649+0000 7f0e817fa700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e841ef480 0x7f0e84148e20 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e840647e0 tx=0x7f0e7c001340 comp rx=0 tx=0).stop 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e7006ad60 msgr2=0x7f0e7006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e7006ad60 0x7f0e7006d220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e840701d0 tx=0x7f0e74008040 comp rx=0 tx=0).stop 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 msgr2=0x7f0e841eea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 --2- 172.21.15.33:0/3998537397 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0e84149ec0 0x7f0e841eea00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e8406f9a0 tx=0x7f0e78003780 comp rx=0 tx=0).stop 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 reap_dead start 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e897d7700 1 -- 172.21.15.33:0/3998537397 reap_dead start 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 shutdown_connections 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 >> 172.21.15.33:0/3998537397 conn(0x7f0e840c8360 msgr2=0x7f0e84070220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.673+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 shutdown_connections 2024-05-08T23:36:36.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:36.674+0000 7f0e8b23a700 1 -- 172.21.15.33:0/3998537397 wait complete. 2024-05-08T23:36:36.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:36.739 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:36:36.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:36:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.048+0000 7f41ff59e700 1 Processor -- start 2024-05-08T23:36:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.049+0000 7f41ff59e700 1 -- start start 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.050+0000 7f41ff59e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f420006bc30 0x7f42001575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.050+0000 7f41ff59e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000c7e60 con 0x7f42001513b0 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.050+0000 7f41ff59e700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4200157b30 con 0x7f420006d430 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.050+0000 7f41ff59e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000c42f0 con 0x7f420006bc30 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.051+0000 7f41fe59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f42001513b0 0x7f420006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:36316/0 (socket says 172.21.15.33:36316) 2024-05-08T23:36:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.051+0000 7f41fe59c700 1 -- 172.21.15.33:0/3577117272 learned_addr learned my addr 172.21.15.33:0/3577117272 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:37.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.051+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949577456 0 0) 0x7f42000c7e60 con 0x7f42001513b0 2024-05-08T23:36:37.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.051+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e80035b0 con 0x7f42001513b0 2024-05-08T23:36:37.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012676572 0 0) 0x7f41e80035b0 con 0x7f42001513b0 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 >> v2:172.21.15.33:3300/0 conn(0x7f420006bc30 msgr2=0x7f42001575f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 --2- 172.21.15.33:0/3577117272 >> v2:172.21.15.33:3300/0 conn(0x7f420006bc30 0x7f42001575f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 >> v1:172.21.15.42:6789/0 conn(0x7f420006d430 legacy=0x7f4200156ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000c7910 con 0x7f42001513b0 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f41f0001670 con 0x7f42001513b0 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 >> v1:172.21.15.33:6789/0 conn(0x7f42001513b0 legacy=0x7f420006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8003a00 0x7f41e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f41e8004850 0x7f41e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 0x7f41e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e800c200 con 0x7f41e8005330 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8014b80 con 0x7f41e8003a00 2024-05-08T23:36:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.052+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8014d40 con 0x7f41e8004850 2024-05-08T23:36:37.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.053+0000 7f41fed9d700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 0x7f41e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:37.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 -- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f41e8004850 msgr2=0x7f41e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:37.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f41e8004850 0x7f41e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 -- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8003a00 msgr2=0x7f41e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:37.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8003a00 0x7f41e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 -- 172.21.15.33:0/3577117272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000c7910 con 0x7f41e8005330 2024-05-08T23:36:37.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.054+0000 7f41fed9d700 1 -- 172.21.15.33:0/3577117272 reap_dead start 2024-05-08T23:36:37.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.056+0000 7f41fed9d700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 0x7f41e800ba10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009e50 tx=0x7f41e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243d18bcbd524475 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:37.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.058+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f80098f0 con 0x7f41e8005330 2024-05-08T23:36:37.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.058+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800bac0 con 0x7f41e8005330 2024-05-08T23:36:37.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.058+0000 7f41fd59a700 1 -- 172.21.15.33:0/3577117272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800be70 con 0x7f41e8005330 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 -- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 msgr2=0x7f41e800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 0x7f41e800ba10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009e50 tx=0x7f41e8005880 comp rx=0 tx=0).stop 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 -- 172.21.15.33:0/3577117272 shutdown_connections 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3577117272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41e8005330 0x7f41e800ba10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 -- 172.21.15.33:0/3577117272 >> 172.21.15.33:0/3577117272 conn(0x7f42000c9370 msgr2=0x7f42000c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 -- 172.21.15.33:0/3577117272 shutdown_connections 2024-05-08T23:36:37.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.059+0000 7f41ff59e700 1 -- 172.21.15.33:0/3577117272 wait complete. 2024-05-08T23:36:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.062+0000 7f41ff59e700 1 Processor -- start 2024-05-08T23:36:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.085+0000 7f41ff59e700 1 -- start start 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42001ec840 0x7f42000c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420006b040 0x7f4200153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000c6c50 con 0x7f420006b040 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42001ebb50 con 0x7f41e8005330 2024-05-08T23:36:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.086+0000 7f41ff59e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42001f0010 con 0x7f42001ec840 2024-05-08T23:36:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.087+0000 7f41fe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.087+0000 7f41fe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:41040/0 (socket says 172.21.15.33:41040) 2024-05-08T23:36:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.087+0000 7f41fe59c700 1 -- 172.21.15.33:0/3901026512 learned_addr learned my addr 172.21.15.33:0/3901026512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 -- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42001ec840 msgr2=0x7f42000c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42001ec840 0x7f42000c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 -- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420006b040 msgr2=0x7f4200153750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420006b040 0x7f4200153750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 -- 172.21.15.33:0/3901026512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000c5210 con 0x7f41e8005330 2024-05-08T23:36:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.088+0000 7f41fe59c700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41f000caa0 tx=0x7f41f000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.092+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004a80 con 0x7f41e8005330 2024-05-08T23:36:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.092+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0004be0 con 0x7f41e8005330 2024-05-08T23:36:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.092+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00070d0 con 0x7f41e8005330 2024-05-08T23:36:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.093+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42000c54f0 con 0x7f41e8005330 2024-05-08T23:36:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.093+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42001ec040 con 0x7f41e8005330 2024-05-08T23:36:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.095+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42000732d0 con 0x7f41e8005330 2024-05-08T23:36:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.097+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f41f0007230 con 0x7f41e8005330 2024-05-08T23:36:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.098+0000 7f41eeffd700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 0x7f41e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:37.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.098+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f41f0088a80 con 0x7f41e8005330 2024-05-08T23:36:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.098+0000 7f41fdd9b700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 0x7f41e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:37.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.099+0000 7f41fdd9b700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 0x7f41e006d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f420014d7d0 tx=0x7f41f4006cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:37.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.106+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f41f0053c80 con 0x7f41e8005330 2024-05-08T23:36:37.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.705+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4200075350 con 0x7f41e8005330 2024-05-08T23:36:37.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.711+0000 7f41eeffd700 1 -- 172.21.15.33:0/3901026512 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f41f0018070 con 0x7f41e8005330 2024-05-08T23:36:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.714+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 msgr2=0x7f41e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.714+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 0x7f41e006d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f420014d7d0 tx=0x7f41f4006cd0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.714+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 msgr2=0x7f42001efad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.714+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f41f000caa0 tx=0x7f41f000ce60 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 shutdown_connections 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f41e006ac90 0x7f41e006d150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420006b040 0x7f4200153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41e8005330 0x7f42001efad0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 --2- 172.21.15.33:0/3901026512 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f42001ec840 0x7f42000c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.715+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 >> 172.21.15.33:0/3901026512 conn(0x7f42000c9370 msgr2=0x7f4200071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.720+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 shutdown_connections 2024-05-08T23:36:37.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:37.721+0000 7f41ff59e700 1 -- 172.21.15.33:0/3901026512 wait complete. 2024-05-08T23:36:37.750 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:36:47.751 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:36:47.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.327+0000 7f8cc68b0700 1 Processor -- start 2024-05-08T23:36:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.327+0000 7f8cc68b0700 1 -- start start 2024-05-08T23:36:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.329+0000 7f8cc68b0700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cc00c2770 0x7f8cc00c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.329+0000 7f8cc68b0700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc006ba40 con 0x7f8cc01589d0 2024-05-08T23:36:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.329+0000 7f8cc68b0700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc0070930 con 0x7f8cc00c7d80 2024-05-08T23:36:48.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.329+0000 7f8cc68b0700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015f180 con 0x7f8cc00c2770 2024-05-08T23:36:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.329+0000 7f8cbf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cc00c2770 0x7f8cc00c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.330+0000 7f8cbf7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8cc00c2770 0x7f8cc00c21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38326/0 (socket says 172.21.15.33:38326) 2024-05-08T23:36:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.330+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/3292043368 learned_addr learned my addr 172.21.15.33:0/3292043368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.330+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/3292043368 >> v1:172.21.15.42:6789/0 conn(0x7f8cc00c7d80 legacy=0x7f8cc0070dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:36:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.330+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/3292043368 >> v1:172.21.15.33:6789/0 conn(0x7f8cc01589d0 legacy=0x7f8cc006a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.330+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/3292043368 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc006b4c0 con 0x7f8cc00c2770 2024-05-08T23:36:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.331+0000 7f8cbf7fe700 1 --2- 172.21.15.33:0/3292043368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c2770 0x7f8cc00c21a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00098f0 tx=0x7f8cb0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578ff4cdfa20130 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.358+0000 7f8cbeffd700 1 -- 172.21.15.33:0/3292043368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004230 con 0x7f8cc00c2770 2024-05-08T23:36:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.358+0000 7f8cbeffd700 1 -- 172.21.15.33:0/3292043368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0004390 con 0x7f8cc00c2770 2024-05-08T23:36:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.358+0000 7f8cbeffd700 1 -- 172.21.15.33:0/3292043368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00036a0 con 0x7f8cc00c2770 2024-05-08T23:36:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.360+0000 7f8cc68b0700 1 -- 172.21.15.33:0/3292043368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c2770 msgr2=0x7f8cc00c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.360+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/3292043368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c2770 0x7f8cc00c21a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00098f0 tx=0x7f8cb0009c00 comp rx=0 tx=0).stop 2024-05-08T23:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.361+0000 7f8cc68b0700 1 -- 172.21.15.33:0/3292043368 shutdown_connections 2024-05-08T23:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.361+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/3292043368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c2770 0x7f8cc00c21a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.361+0000 7f8cc68b0700 1 -- 172.21.15.33:0/3292043368 >> 172.21.15.33:0/3292043368 conn(0x7f8cc00c8360 msgr2=0x7f8cc00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.372+0000 7f8cc68b0700 1 -- 172.21.15.33:0/3292043368 shutdown_connections 2024-05-08T23:36:48.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.373+0000 7f8cc68b0700 1 -- 172.21.15.33:0/3292043368 wait complete. 2024-05-08T23:36:48.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.375+0000 7f8cc68b0700 1 Processor -- start 2024-05-08T23:36:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.387+0000 7f8cc68b0700 1 -- start start 2024-05-08T23:36:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cc00c2770 0x7f8cc015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8cc01589d0 0x7f8cc0146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc015a300 con 0x7f8cc00c7d80 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01409e0 con 0x7f8cc00c2770 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.388+0000 7f8cc68b0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0142730 con 0x7f8cc01589d0 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38328/0 (socket says 172.21.15.33:38328) 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/1434470091 learned_addr learned my addr 172.21.15.33:0/1434470091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbffff700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cc00c2770 0x7f8cc015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8cc01589d0 msgr2=0x7f8cc0146bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8cc01589d0 0x7f8cc0146bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cc00c2770 msgr2=0x7f8cc015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cc00c2770 0x7f8cc015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 -- 172.21.15.33:0/1434470091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0009550 con 0x7f8cc00c7d80 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.389+0000 7f8cbf7fe700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8cc006f5d0 tx=0x7f8cb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.390+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011630 con 0x7f8cc00c7d80 2024-05-08T23:36:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.390+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0011790 con 0x7f8cc00c7d80 2024-05-08T23:36:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.390+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011b40 con 0x7f8cc00c7d80 2024-05-08T23:36:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.390+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0140c50 con 0x7f8cc00c7d80 2024-05-08T23:36:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.390+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc0141210 con 0x7f8cc00c7d80 2024-05-08T23:36:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.391+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc0075460 con 0x7f8cc00c7d80 2024-05-08T23:36:48.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.403+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8cb0043070 con 0x7f8cc00c7d80 2024-05-08T23:36:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.404+0000 7f8cbcff9700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 0x7f8cac06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.404+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8cb00acb50 con 0x7f8cc00c7d80 2024-05-08T23:36:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.404+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f8cb00b0280 con 0x7f8cc00c7d80 2024-05-08T23:36:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.405+0000 7f8cbffff700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 0x7f8cac06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:48.406+0000 7f8cbffff700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 0x7f8cac06d180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b450 tx=0x7f8cb800d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:48.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:36:48.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:36:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.181+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8cc0075460 con 0x7f8cc00c7d80 2024-05-08T23:36:49.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.181+0000 7f8cbcff9700 1 -- 172.21.15.33:0/1434470091 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f8cb004b020 con 0x7f8cc00c7d80 2024-05-08T23:36:49.180 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 msgr2=0x7f8cac06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 0x7f8cac06d180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b450 tx=0x7f8cb800d040 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 msgr2=0x7f8cc00c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8cc006f5d0 tx=0x7f8cb0004750 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 shutdown_connections 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8cac06acc0 0x7f8cac06d180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8cc01589d0 0x7f8cc0146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8cc00c2770 0x7f8cc015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 --2- 172.21.15.33:0/1434470091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cc00c7d80 0x7f8cc00c61c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.185+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 >> 172.21.15.33:0/1434470091 conn(0x7f8cc00c8360 msgr2=0x7f8cc00c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:49.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.186+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 shutdown_connections 2024-05-08T23:36:49.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.187+0000 7f8cc68b0700 1 -- 172.21.15.33:0/1434470091 wait complete. 2024-05-08T23:36:49.228 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:36:49.228 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:36:49.228 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:36:49.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:36:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.588+0000 7fe700df2700 1 Processor -- start 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.589+0000 7fe700df2700 1 -- start start 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.590+0000 7fe700df2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c70c0 0x7fe6fc0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.590+0000 7fe700df2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc157c30 con 0x7fe6fc1577a0 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.590+0000 7fe700df2700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc0c8110 con 0x7fe6fc150510 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.590+0000 7fe700df2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0c7e30 con 0x7fe6fc0c70c0 2024-05-08T23:36:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.591+0000 7fe6fad9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe6fc1577a0 0x7fe6fc160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60780/0 (socket says 172.21.15.33:60780) 2024-05-08T23:36:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.591+0000 7fe6fad9d700 1 -- 172.21.15.33:0/2935654586 learned_addr learned my addr 172.21.15.33:0/2935654586 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:36:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670634437 0 0) 0x7fe6fc157c30 con 0x7fe6fc1577a0 2024-05-08T23:36:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e40035b0 con 0x7fe6fc1577a0 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1570268474 0 0) 0x7fe6e40035b0 con 0x7fe6fc1577a0 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c70c0 msgr2=0x7fe6fc0c7780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c70c0 0x7fe6fc0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 >> v1:172.21.15.42:6789/0 conn(0x7fe6fc150510 legacy=0x7fe6fc06a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0c6300 con 0x7fe6fc1577a0 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fe6f40019f0 con 0x7fe6fc1577a0 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 >> v1:172.21.15.33:6789/0 conn(0x7fe6fc1577a0 legacy=0x7fe6fc160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4003a30 0x7fe6e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6e4004850 0x7fe6e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6e4005330 0x7fe6e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0c7e30 con 0x7fe6e4003a30 2024-05-08T23:36:49.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e400c180 con 0x7fe6e4004850 2024-05-08T23:36:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.592+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e400c340 con 0x7fe6e4005330 2024-05-08T23:36:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.595+0000 7fe6f9d9b700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6e4004850 0x7fe6e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.599+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6e4005330 0x7fe6e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.601+0000 7fe6fad9d700 1 -- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6e4004850 msgr2=0x7fe6e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:36:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.601+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6e4004850 0x7fe6e4004cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.602+0000 7fe6fad9d700 1 -- 172.21.15.33:0/2935654586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4003a30 msgr2=0x7fe6e4003e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.602+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4003a30 0x7fe6e4003e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.602+0000 7fe6fad9d700 1 -- 172.21.15.33:0/2935654586 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0c6300 con 0x7fe6e4005330 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.604+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6e4005330 0x7fe6e400ba10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40054d0 tx=0x7fe6e4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e080a0ceb784146 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.607+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4008310 con 0x7fe6e4005330 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.607+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4004ec0 con 0x7fe6e4005330 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.607+0000 7fe6f959a700 1 -- 172.21.15.33:0/2935654586 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40067b0 con 0x7fe6e4005330 2024-05-08T23:36:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.608+0000 7fe700df2700 1 -- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6e4005330 msgr2=0x7fe6e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.608+0000 7fe700df2700 1 --2- 172.21.15.33:0/2935654586 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6e4005330 0x7fe6e400ba10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40054d0 tx=0x7fe6e4005880 comp rx=0 tx=0).stop 2024-05-08T23:36:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.609+0000 7fe6fad9d700 1 -- 172.21.15.33:0/2935654586 reap_dead start 2024-05-08T23:36:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.609+0000 7fe700df2700 1 -- 172.21.15.33:0/2935654586 shutdown_connections 2024-05-08T23:36:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.609+0000 7fe700df2700 1 -- 172.21.15.33:0/2935654586 >> 172.21.15.33:0/2935654586 conn(0x7fe6fc0c8360 msgr2=0x7fe6fc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.610+0000 7fe700df2700 1 -- 172.21.15.33:0/2935654586 shutdown_connections 2024-05-08T23:36:49.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.620+0000 7fe700df2700 1 -- 172.21.15.33:0/2935654586 wait complete. 2024-05-08T23:36:49.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.622+0000 7fe700df2700 1 Processor -- start 2024-05-08T23:36:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.623+0000 7fe700df2700 1 -- start start 2024-05-08T23:36:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4005330 0x7fe6fc152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 0x7fe6fc0714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c4650 0x7fe6fc0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc071060 con 0x7fe6e4005330 2024-05-08T23:36:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc150c80 con 0x7fe6fc153b80 2024-05-08T23:36:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.624+0000 7fe700df2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc1ec3d0 con 0x7fe6fc0c4650 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.625+0000 7fe6f9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 0x7fe6fc0714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.625+0000 7fe6f9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 0x7fe6fc0714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:56244/0 (socket says 172.21.15.33:56244) 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.625+0000 7fe6f9d9b700 1 -- 172.21.15.33:0/4286803289 learned_addr learned my addr 172.21.15.33:0/4286803289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c4650 0x7fe6fc0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6fa59c700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4005330 0x7fe6fc152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6f9d9b700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c4650 msgr2=0x7fe6fc0719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6f9d9b700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe6fc0c4650 0x7fe6fc0719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6f9d9b700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4005330 msgr2=0x7fe6fc152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6f9d9b700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e4005330 0x7fe6fc152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.626+0000 7fe6f9d9b700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc1ec6b0 con 0x7fe6fc153b80 2024-05-08T23:36:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.628+0000 7fe6f9d9b700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 0x7fe6fc0714b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c570 tx=0x7fe6ec00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.630+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec011800 con 0x7fe6fc153b80 2024-05-08T23:36:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.630+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec011980 con 0x7fe6fc153b80 2024-05-08T23:36:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.630+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec011d30 con 0x7fe6fc153b80 2024-05-08T23:36:49.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.633+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc150ed0 con 0x7fe6fc153b80 2024-05-08T23:36:49.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.633+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc1524d0 con 0x7fe6fc153b80 2024-05-08T23:36:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.638+0000 7fe700df2700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 0x7fe6fc1ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.638+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc074a20 con 0x7fe6fc1eee20 2024-05-08T23:36:49.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.640+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe6ec022350 con 0x7fe6fc153b80 2024-05-08T23:36:49.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.641+0000 7fe6fa59c700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 0x7fe6fc1ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.641+0000 7fe6eaffd700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe6dc06ad90 0x7fe6dc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.641+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fe6ec089160 con 0x7fe6fc153b80 2024-05-08T23:36:49.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.641+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe6dc06ad90 0x7fe6dc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.641+0000 7fe6fa59c700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6fc0c6ba0 con 0x7fe6fc1eee20 2024-05-08T23:36:49.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.642+0000 7fe6fa59c700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 0x7fe6fc1ef200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0013680 con 0x7fe6fc1eee20 2024-05-08T23:36:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe6f00137e0 con 0x7fe6fc1eee20 2024-05-08T23:36:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 msgr2=0x7fe6fc1ef200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 0x7fe6fc1ef200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f000eb60 comp rx=0 tx=0).stop 2024-05-08T23:36:49.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 msgr2=0x7fe6fc1ef200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:49.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.644+0000 7fe6eaffd700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1eee20 0x7fe6fc1ef200 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f000eb60 comp rx=0 tx=0).stop 2024-05-08T23:36:49.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.646+0000 7fe6fad9d700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe6dc06ad90 0x7fe6dc06d250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc06d4f0 tx=0x7fe6f4007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.659+0000 7fe700df2700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 0x7fe6fc14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.659+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc151060 con 0x7fe6fc1ef740 2024-05-08T23:36:49.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.659+0000 7fe6fa59c700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 0x7fe6fc14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:49.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.671+0000 7fe6fa59c700 1 -- 172.21.15.33:0/4286803289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe6fc0c68c0 con 0x7fe6fc1ef740 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.671+0000 7fe6fa59c700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 0x7fe6fc14ac40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f0004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0021070 con 0x7fe6fc1ef740 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5113 (secure 0 0 0) 0x7fe6f0006490 con 0x7fe6fc1ef740 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 msgr2=0x7fe6fc14ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 0x7fe6fc14ac40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f0004800 comp rx=0 tx=0).stop 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 msgr2=0x7fe6fc14ac40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:49.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.672+0000 7fe6eaffd700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6fc1ef740 0x7fe6fc14ac40 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000e7a0 tx=0x7fe6f0004800 comp rx=0 tx=0).stop 2024-05-08T23:36:49.679 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:49.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 19, 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:49.680 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:49.681 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:49.682 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:49.683 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:49.684 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:49.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:49.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:49.689 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:49.691 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:49.692 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.689+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe6dc06ad90 msgr2=0x7fe6dc06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.689+0000 7fe700df2700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe6dc06ad90 0x7fe6dc06d250 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc06d4f0 tx=0x7fe6f4007000 comp rx=0 tx=0).stop 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.689+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 msgr2=0x7fe6fc0714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.689+0000 7fe700df2700 1 --2- 172.21.15.33:0/4286803289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6fc153b80 0x7fe6fc0714b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00c570 tx=0x7fe6ec00c880 comp rx=0 tx=0).stop 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.690+0000 7fe6fad9d700 1 -- 172.21.15.33:0/4286803289 reap_dead start 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.690+0000 7fe6fad9d700 1 -- 172.21.15.33:0/4286803289 reap_dead start 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.691+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 shutdown_connections 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.691+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 >> 172.21.15.33:0/4286803289 conn(0x7fe6fc0c8360 msgr2=0x7fe6fc157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.691+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 shutdown_connections 2024-05-08T23:36:49.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:49.691+0000 7fe700df2700 1 -- 172.21.15.33:0/4286803289 wait complete. 2024-05-08T23:36:49.721 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:49.721 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:36:49.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:36:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.511+0000 7f236884b700 1 Processor -- start 2024-05-08T23:36:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.512+0000 7f236884b700 1 -- start start 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.513+0000 7f236884b700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23640c8e50 0x7f23640c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.513+0000 7f236884b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2364157c30 con 0x7f23641577a0 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.513+0000 7f236884b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23640c8110 con 0x7f23640c70c0 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.513+0000 7f236884b700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23640c7e30 con 0x7f23640c8e50 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.514+0000 7f236259c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23640c8e50 0x7f23640c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.514+0000 7f236259c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f23640c8e50 0x7f23640c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:56250/0 (socket says 172.21.15.33:56250) 2024-05-08T23:36:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.514+0000 7f236259c700 1 -- 172.21.15.33:0/2858646383 learned_addr learned my addr 172.21.15.33:0/2858646383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.521+0000 7f236259c700 1 -- 172.21.15.33:0/2858646383 >> v1:172.21.15.42:6789/0 conn(0x7f23640c70c0 legacy=0x7f2364150480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:36:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.521+0000 7f236259c700 1 -- 172.21.15.33:0/2858646383 >> v1:172.21.15.33:6789/0 conn(0x7f23641577a0 legacy=0x7f236406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.521+0000 7f236259c700 1 -- 172.21.15.33:0/2858646383 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640c6300 con 0x7f23640c8e50 2024-05-08T23:36:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.522+0000 7f236259c700 1 --2- 172.21.15.33:0/2858646383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 0x7f23640c7780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23540131e0 tx=0x7f23540134f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8707d5a40ad0a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.541+0000 7f236159a700 1 -- 172.21.15.33:0/2858646383 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354016070 con 0x7f23640c8e50 2024-05-08T23:36:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.541+0000 7f236159a700 1 -- 172.21.15.33:0/2858646383 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2354013d30 con 0x7f23640c8e50 2024-05-08T23:36:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.541+0000 7f236159a700 1 -- 172.21.15.33:0/2858646383 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354004230 con 0x7f23640c8e50 2024-05-08T23:36:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.542+0000 7f236884b700 1 -- 172.21.15.33:0/2858646383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 msgr2=0x7f23640c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.542+0000 7f236884b700 1 --2- 172.21.15.33:0/2858646383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 0x7f23640c7780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23540131e0 tx=0x7f23540134f0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.546+0000 7f236884b700 1 -- 172.21.15.33:0/2858646383 shutdown_connections 2024-05-08T23:36:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.546+0000 7f236884b700 1 --2- 172.21.15.33:0/2858646383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 0x7f23640c7780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.546+0000 7f236884b700 1 -- 172.21.15.33:0/2858646383 >> 172.21.15.33:0/2858646383 conn(0x7f23640c8360 msgr2=0x7f23640c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.552+0000 7f236884b700 1 -- 172.21.15.33:0/2858646383 shutdown_connections 2024-05-08T23:36:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.552+0000 7f236884b700 1 -- 172.21.15.33:0/2858646383 wait complete. 2024-05-08T23:36:50.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.554+0000 7f236884b700 1 Processor -- start 2024-05-08T23:36:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.555+0000 7f236884b700 1 -- start start 2024-05-08T23:36:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.556+0000 7f236884b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f23640c70c0 0x7f236414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.556+0000 7f236884b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 0x7f23641eba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.556+0000 7f236884b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 0x7f2364156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f2362d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 0x7f2364156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f2362d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 0x7f2364156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38360/0 (socket says 172.21.15.33:38360) 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 learned_addr learned my addr 172.21.15.33:0/4153880899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23641eb5f0 con 0x7f23641577a0 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640c34d0 con 0x7f23640c8e50 2024-05-08T23:36:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.557+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23641eb2e0 con 0x7f23640c70c0 2024-05-08T23:36:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f23640c70c0 msgr2=0x7f236414d090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f23640c70c0 0x7f236414d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 msgr2=0x7f23641eba80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23640c8e50 0x7f23641eba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2354011df0 con 0x7f23641577a0 2024-05-08T23:36:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.560+0000 7f2362d9d700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 0x7f2364156a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cb20 tx=0x7f235c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:50.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.585+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c003e30 con 0x7f23641577a0 2024-05-08T23:36:50.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.585+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c005070 con 0x7f23641577a0 2024-05-08T23:36:50.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.585+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c005400 con 0x7f23641577a0 2024-05-08T23:36:50.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.585+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2364148d30 con 0x7f23641577a0 2024-05-08T23:36:50.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.585+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2364149290 con 0x7f23641577a0 2024-05-08T23:36:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.591+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f235c005560 con 0x7f23641577a0 2024-05-08T23:36:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.592+0000 7f2352ffd700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f234406ad90 0x7f234406d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.592+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f235c088c60 con 0x7f23641577a0 2024-05-08T23:36:50.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.594+0000 7f236259c700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f234406ad90 0x7f234406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:50.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.595+0000 7f236259c700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f234406ad90 0x7f234406d250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354011000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.606+0000 7f236884b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 0x7f2348004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.606+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2348007f20 con 0x7f2348002a40 2024-05-08T23:36:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.613+0000 7f2361d9b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 0x7f2348004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.614+0000 7f2361d9b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23480080e0 con 0x7f2348002a40 2024-05-08T23:36:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.614+0000 7f2361d9b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 0x7f2348004f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f235800b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358011620 con 0x7f2348002a40 2024-05-08T23:36:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2358011780 con 0x7f2348002a40 2024-05-08T23:36:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 msgr2=0x7f2348004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 0x7f2348004f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f235800b6a0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 msgr2=0x7f2348004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.616+0000 7f2352ffd700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348002a40 0x7f2348004f00 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f235800b6a0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.633+0000 7f236884b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 0x7f2348007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:50.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.633+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234800a3e0 con 0x7f2348006fc0 2024-05-08T23:36:50.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.634+0000 7f2361d9b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 0x7f2348007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.638+0000 7f2361d9b700 1 -- 172.21.15.33:0/4153880899 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f234800a5a0 con 0x7f2348006fc0 2024-05-08T23:36:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.638+0000 7f2361d9b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 0x7f2348007450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f23580046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235801ca70 con 0x7f2348006fc0 2024-05-08T23:36:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4968 (secure 0 0 0) 0x7f235801a420 con 0x7f2348006fc0 2024-05-08T23:36:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 msgr2=0x7f2348007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 0x7f2348007450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f23580046d0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 msgr2=0x7f2348007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.642+0000 7f2352ffd700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2348006fc0 0x7f2348007450 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f236414e380 tx=0x7f23580046d0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.676 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 20, 2024-05-08T23:36:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:50.679 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:50.680 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:50.684 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:50.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:50.686 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.687 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:50.688 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:50.689 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.706+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f234406ad90 msgr2=0x7f234406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f236884b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f234406ad90 0x7f234406d250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354011000 comp rx=0 tx=0).stop 2024-05-08T23:36:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 msgr2=0x7f2364156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:50.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f236884b700 1 --2- 172.21.15.33:0/4153880899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641577a0 0x7f2364156a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cb20 tx=0x7f235c00cee0 comp rx=0 tx=0).stop 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 reap_dead start 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f2362d9d700 1 -- 172.21.15.33:0/4153880899 reap_dead start 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 shutdown_connections 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.707+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 >> 172.21.15.33:0/4153880899 conn(0x7f23640c8360 msgr2=0x7f23640caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.708+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 shutdown_connections 2024-05-08T23:36:50.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:50.708+0000 7f236884b700 1 -- 172.21.15.33:0/4153880899 wait complete. 2024-05-08T23:36:50.732 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:50.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:36:50.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.118+0000 7f1596c12700 1 Processor -- start 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.118+0000 7f1596c12700 1 -- start start 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.119+0000 7f1596c12700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15900c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.119+0000 7f1596c12700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15900c7d80 0x7f1590070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.119+0000 7f1596c12700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159006ba40 con 0x7f15901589d0 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.119+0000 7f1596c12700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159015f180 con 0x7f15900c7d80 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.119+0000 7f1596c12700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900c8210 con 0x7f15900c2770 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15900c7d80 0x7f1590070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15900c7d80 0x7f1590070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38370/0 (socket says 172.21.15.33:38370) 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 -- 172.21.15.33:0/3451175304 learned_addr learned my addr 172.21.15.33:0/3451175304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 -- 172.21.15.33:0/3451175304 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 msgr2=0x7f15900c21a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 --2- 172.21.15.33:0/3451175304 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15900c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 -- 172.21.15.33:0/3451175304 >> v1:172.21.15.42:6789/0 conn(0x7f15901589d0 legacy=0x7f159006a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.120+0000 7f15951af700 1 -- 172.21.15.33:0/3451175304 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590070930 con 0x7f15900c7d80 2024-05-08T23:36:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.121+0000 7f15951af700 1 --2- 172.21.15.33:0/3451175304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f1590070dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ee0243d7f6d15f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.121+0000 7f158f7fe700 1 -- 172.21.15.33:0/3451175304 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e070 con 0x7f15900c7d80 2024-05-08T23:36:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.121+0000 7f158f7fe700 1 -- 172.21.15.33:0/3451175304 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588003a20 con 0x7f15900c7d80 2024-05-08T23:36:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.121+0000 7f158f7fe700 1 -- 172.21.15.33:0/3451175304 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588003dd0 con 0x7f15900c7d80 2024-05-08T23:36:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.123+0000 7f1596c12700 1 -- 172.21.15.33:0/3451175304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 msgr2=0x7f1590070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.123+0000 7f1596c12700 1 --2- 172.21.15.33:0/3451175304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f1590070dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).stop 2024-05-08T23:36:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 -- 172.21.15.33:0/3451175304 shutdown_connections 2024-05-08T23:36:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 --2- 172.21.15.33:0/3451175304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f1590070dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 --2- 172.21.15.33:0/3451175304 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15900c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 -- 172.21.15.33:0/3451175304 >> 172.21.15.33:0/3451175304 conn(0x7f15900c8360 msgr2=0x7f15900c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 -- 172.21.15.33:0/3451175304 shutdown_connections 2024-05-08T23:36:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.124+0000 7f1596c12700 1 -- 172.21.15.33:0/3451175304 wait complete. 2024-05-08T23:36:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.126+0000 7f1596c12700 1 Processor -- start 2024-05-08T23:36:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.126+0000 7f1596c12700 1 -- start start 2024-05-08T23:36:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15901350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f15900c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 0x7f159015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159015c070 con 0x7f15900c7d80 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159015afc0 con 0x7f15901589d0 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.128+0000 7f1596c12700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1590159780 con 0x7f15900c2770 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.129+0000 7f15951af700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 0x7f159015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.129+0000 7f15951af700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 0x7f159015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:56260/0 (socket says 172.21.15.33:56260) 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.129+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 learned_addr learned my addr 172.21.15.33:0/41341434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.129+0000 7f15949ae700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15901350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.129+0000 7f158ffff700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f15900c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.130+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 msgr2=0x7f15901350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.130+0000 7f15951af700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f15900c2770 0x7f15901350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.130+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 msgr2=0x7f15900c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.130+0000 7f15951af700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15900c7d80 0x7f15900c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.130+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588009b30 con 0x7f15901589d0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f15951af700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 0x7f159015b450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f159006f5d0 tx=0x7f1588004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004440 con 0x7f15901589d0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15880045c0 con 0x7f15901589d0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004970 con 0x7f15901589d0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1590136010 con 0x7f15901589d0 2024-05-08T23:36:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.131+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1590135a50 con 0x7f15901589d0 2024-05-08T23:36:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.133+0000 7f1596c12700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 0x7f156c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.133+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c007f20 con 0x7f156c002a40 2024-05-08T23:36:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.134+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1588012430 con 0x7f15901589d0 2024-05-08T23:36:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.134+0000 7f158d7fa700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f157406acc0 0x7f157406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.134+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f158808a750 con 0x7f15901589d0 2024-05-08T23:36:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.135+0000 7f158ffff700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f157406acc0 0x7f157406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.136+0000 7f158ffff700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f157406acc0 0x7f157406d180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15900c7a50 tx=0x7f157c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.137+0000 7f15949ae700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 0x7f156c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.140+0000 7f15949ae700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f156c0080e0 con 0x7f156c002a40 2024-05-08T23:36:51.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.146+0000 7f15949ae700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 0x7f156c004f00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f158400d730 tx=0x7f158400daf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840125f0 con 0x7f156c002a40 2024-05-08T23:36:51.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1584012750 con 0x7f156c002a40 2024-05-08T23:36:51.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 msgr2=0x7f156c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 0x7f156c004f00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f158400d730 tx=0x7f158400daf0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 msgr2=0x7f156c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:51.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.151+0000 7f158d7fa700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c002a40 0x7f156c004f00 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f158400d730 tx=0x7f158400daf0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.168+0000 7f1596c12700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 0x7f156c007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.168+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00a3e0 con 0x7f156c006fc0 2024-05-08T23:36:51.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.168+0000 7f15949ae700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 0x7f156c007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.169+0000 7f15949ae700 1 -- 172.21.15.33:0/41341434 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f156c00a5a0 con 0x7f156c006fc0 2024-05-08T23:36:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.172+0000 7f15949ae700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 0x7f156c007450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f158400b150 tx=0x7f1584004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.173+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158401da30 con 0x7f156c006fc0 2024-05-08T23:36:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.177+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4458 (secure 0 0 0) 0x7f1584004020 con 0x7f156c006fc0 2024-05-08T23:36:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.177+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 msgr2=0x7f156c007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.177+0000 7f158d7fa700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 0x7f156c007450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f158400b150 tx=0x7f1584004800 comp rx=0 tx=0).stop 2024-05-08T23:36:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.177+0000 7f158d7fa700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 msgr2=0x7f156c007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:36:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.177+0000 7f158d7fa700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f156c006fc0 0x7f156c007450 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f158400b150 tx=0x7f1584004800 comp rx=0 tx=0).stop 2024-05-08T23:36:51.183 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:36:51.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 38, 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 20, 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:36:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:36:51.186 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:36:51.187 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.188 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.189 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:51.190 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:36:51.192 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:36:51.194 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.194+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f157406acc0 msgr2=0x7f157406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.194+0000 7f1596c12700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f157406acc0 0x7f157406d180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15900c7a50 tx=0x7f157c008040 comp rx=0 tx=0).stop 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.194+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 msgr2=0x7f159015b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.194+0000 7f1596c12700 1 --2- 172.21.15.33:0/41341434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15901589d0 0x7f159015b450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f159006f5d0 tx=0x7f1588004ad0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.196+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 reap_dead start 2024-05-08T23:36:51.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.196+0000 7f15951af700 1 -- 172.21.15.33:0/41341434 reap_dead start 2024-05-08T23:36:51.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.202+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 shutdown_connections 2024-05-08T23:36:51.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.202+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 >> 172.21.15.33:0/41341434 conn(0x7f15900c8360 msgr2=0x7f159014aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:51.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.208+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 shutdown_connections 2024-05-08T23:36:51.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.209+0000 7f1596c12700 1 -- 172.21.15.33:0/41341434 wait complete. 2024-05-08T23:36:51.255 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:36:51.255 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-05-08T23:36:51.256 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:36:51.256 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-05-08T23:36:51.256 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T23:36:51.256 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T23:36:51.256 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:36:51.276 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T23:36:51.276 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:36:51.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.664+0000 7f1eee473700 1 Processor -- start 2024-05-08T23:36:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.664+0000 7f1eee473700 1 -- start start 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1eee473700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 0x7f1ee8072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1eee473700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ee815af30 0x7f1ee8072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1eee473700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee806ce60 con 0x7f1ee80b2b60 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1eee473700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee815b470 con 0x7f1ee815af30 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1eee473700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80c90a0 con 0x7f1ee81511c0 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1ee7fff700 1 -- >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 msgr2=0x7f1ee8072400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.665+0000 7f1ee7fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 0x7f1ee8072400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:36:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.666+0000 7f1eeca10700 1 -- >> v1:172.21.15.42:6789/0 conn(0x7f1ee80b2b60 legacy=0x7f1ee8156f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:36:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.667+0000 7f1ee77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ee815af30 0x7f1ee8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.667+0000 7f1ee77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ee815af30 0x7f1ee8072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38408/0 (socket says 172.21.15.33:38408) 2024-05-08T23:36:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.667+0000 7f1ee77fe700 1 -- 172.21.15.33:0/4281498090 learned_addr learned my addr 172.21.15.33:0/4281498090 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.671+0000 7f1ee77fe700 1 -- 172.21.15.33:0/4281498090 >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 msgr2=0x7f1ee8072400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.671+0000 7f1ee77fe700 1 --2- 172.21.15.33:0/4281498090 >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 0x7f1ee8072400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.671+0000 7f1ee77fe700 1 -- 172.21.15.33:0/4281498090 >> v1:172.21.15.42:6789/0 conn(0x7f1ee80b2b60 legacy=0x7f1ee8156f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.671+0000 7f1ee77fe700 1 -- 172.21.15.33:0/4281498090 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80c9380 con 0x7f1ee815af30 2024-05-08T23:36:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.671+0000 7f1ee77fe700 1 --2- 172.21.15.33:0/4281498090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee815af30 0x7f1ee8072940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80ca7e0 tx=0x7f1edc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5b902be33ee07e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.675+0000 7f1ee6ffd700 1 -- 172.21.15.33:0/4281498090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00dcf0 con 0x7f1ee815af30 2024-05-08T23:36:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.675+0000 7f1ee6ffd700 1 -- 172.21.15.33:0/4281498090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00de50 con 0x7f1ee815af30 2024-05-08T23:36:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.675+0000 7f1ee6ffd700 1 -- 172.21.15.33:0/4281498090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc004250 con 0x7f1ee815af30 2024-05-08T23:36:51.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.677+0000 7f1eee473700 1 -- 172.21.15.33:0/4281498090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee815af30 msgr2=0x7f1ee8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.677+0000 7f1eee473700 1 --2- 172.21.15.33:0/4281498090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee815af30 0x7f1ee8072940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80ca7e0 tx=0x7f1edc00be70 comp rx=0 tx=0).stop 2024-05-08T23:36:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.685+0000 7f1eee473700 1 -- 172.21.15.33:0/4281498090 shutdown_connections 2024-05-08T23:36:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.685+0000 7f1eee473700 1 --2- 172.21.15.33:0/4281498090 >> v2:172.21.15.42:3300/0 conn(0x7f1ee81511c0 0x7f1ee8072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.685+0000 7f1eee473700 1 --2- 172.21.15.33:0/4281498090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee815af30 0x7f1ee8072940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.685+0000 7f1eee473700 1 -- 172.21.15.33:0/4281498090 >> 172.21.15.33:0/4281498090 conn(0x7f1ee806e990 msgr2=0x7f1ee806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:51.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.686+0000 7f1eee473700 1 -- 172.21.15.33:0/4281498090 shutdown_connections 2024-05-08T23:36:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.687+0000 7f1eee473700 1 -- 172.21.15.33:0/4281498090 wait complete. 2024-05-08T23:36:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.689+0000 7f1eee473700 1 Processor -- start 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.690+0000 7f1eee473700 1 -- start start 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ee81511c0 0x7f1ee806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ee815af30 0x7f1ee81580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8155190 con 0x7f1ee80b2b60 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8156e40 con 0x7f1ee81511c0 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1eee473700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee815a0c0 con 0x7f1ee815af30 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38412/0 (socket says 172.21.15.33:38412) 2024-05-08T23:36:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.691+0000 7f1ee7fff700 1 -- 172.21.15.33:0/1561278157 learned_addr learned my addr 172.21.15.33:0/1561278157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1eeca10700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ee815af30 0x7f1ee81580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1ee7fff700 1 -- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ee815af30 msgr2=0x7f1ee81580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1ee7fff700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ee815af30 0x7f1ee81580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1ee7fff700 1 -- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ee81511c0 msgr2=0x7f1ee806bc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1ee7fff700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ee81511c0 0x7f1ee806bc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.692+0000 7f1ee7fff700 1 -- 172.21.15.33:0/1561278157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc00b7f0 con 0x7f1ee80b2b60 2024-05-08T23:36:51.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.694+0000 7f1ee7fff700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b7b0 tx=0x7f1ed800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.711+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8003e50 con 0x7f1ee80b2b60 2024-05-08T23:36:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.711+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8005070 con 0x7f1ee80b2b60 2024-05-08T23:36:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.711+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005420 con 0x7f1ee80b2b60 2024-05-08T23:36:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.714+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee81570b0 con 0x7f1ee80b2b60 2024-05-08T23:36:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.714+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee8157390 con 0x7f1ee80b2b60 2024-05-08T23:36:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.715+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee8075460 con 0x7f1ee80b2b60 2024-05-08T23:36:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.728+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1ed8005580 con 0x7f1ee80b2b60 2024-05-08T23:36:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.728+0000 7f1ee4ff9700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 0x7f1ed406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.729+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1ed8089ba0 con 0x7f1ee80b2b60 2024-05-08T23:36:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.729+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1ed80b69a0 con 0x7f1ee80b2b60 2024-05-08T23:36:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.729+0000 7f1ee77fe700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 0x7f1ed406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:51.735+0000 7f1ee77fe700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 0x7f1ed406d220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee815b3a0 tx=0x7f1edc012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.597+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1ee8150ad0 con 0x7f1ee80b2b60 2024-05-08T23:36:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.598+0000 7f1ee4ff9700 1 -- 172.21.15.33:0/1561278157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f1ed8054c00 con 0x7f1ee80b2b60 2024-05-08T23:36:52.601 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 msgr2=0x7f1ed406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 0x7f1ed406d220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ee815b3a0 tx=0x7f1edc012000 comp rx=0 tx=0).stop 2024-05-08T23:36:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 msgr2=0x7f1ee81e04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b7b0 tx=0x7f1ed800bb70 comp rx=0 tx=0).stop 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 shutdown_connections 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1ed406ad60 0x7f1ed406d220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ee815af30 0x7f1ee81580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee80b2b60 0x7f1ee81e04f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 --2- 172.21.15.33:0/1561278157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ee81511c0 0x7f1ee806bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 >> 172.21.15.33:0/1561278157 conn(0x7f1ee806e990 msgr2=0x7f1ee80c8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 shutdown_connections 2024-05-08T23:36:52.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:52.607+0000 7f1eee473700 1 -- 172.21.15.33:0/1561278157 wait complete. 2024-05-08T23:36:55.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:36:56.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.133+0000 7f57bbca4700 1 Processor -- start 2024-05-08T23:36:56.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.133+0000 7f57bbca4700 1 -- start start 2024-05-08T23:36:56.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57bbca4700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57b40c58c0 0x7f57b4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57bbca4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b4070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57bbca4700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b4072e40 con 0x7f57b41670c0 2024-05-08T23:36:56.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57bbca4700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b406b920 con 0x7f57b40c58c0 2024-05-08T23:36:56.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57bbca4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b406aa30 con 0x7f57b40c9050 2024-05-08T23:36:56.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57b9a40700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57b40c58c0 0x7f57b4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:56.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57b9a40700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f57b40c58c0 0x7f57b4070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54366/0 (socket says 172.21.15.33:54366) 2024-05-08T23:36:56.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.135+0000 7f57b9a40700 1 -- 172.21.15.33:0/2696135611 learned_addr learned my addr 172.21.15.33:0/2696135611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.136+0000 7f57b9a40700 1 -- 172.21.15.33:0/2696135611 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 msgr2=0x7f57b4070550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.136+0000 7f57b9a40700 1 --2- 172.21.15.33:0/2696135611 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b4070550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.136+0000 7f57b9a40700 1 -- 172.21.15.33:0/2696135611 >> v1:172.21.15.42:6789/0 conn(0x7f57b41670c0 legacy=0x7f57b4070cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.136+0000 7f57b9a40700 1 -- 172.21.15.33:0/2696135611 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b406ad10 con 0x7f57b40c58c0 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.136+0000 7f57b9a40700 1 --2- 172.21.15.33:0/2696135611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b4070010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098f0 tx=0x7f57b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e007ea22d1cd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.138+0000 7f57b8a3e700 1 -- 172.21.15.33:0/2696135611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004230 con 0x7f57b40c58c0 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.138+0000 7f57b8a3e700 1 -- 172.21.15.33:0/2696135611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0004390 con 0x7f57b40c58c0 2024-05-08T23:36:56.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.138+0000 7f57b8a3e700 1 -- 172.21.15.33:0/2696135611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00036a0 con 0x7f57b40c58c0 2024-05-08T23:36:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.139+0000 7f57bbca4700 1 -- 172.21.15.33:0/2696135611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 msgr2=0x7f57b4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.139+0000 7f57bbca4700 1 --2- 172.21.15.33:0/2696135611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b4070010 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098f0 tx=0x7f57b0009c00 comp rx=0 tx=0).stop 2024-05-08T23:36:56.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.147+0000 7f57bbca4700 1 -- 172.21.15.33:0/2696135611 shutdown_connections 2024-05-08T23:36:56.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.147+0000 7f57bbca4700 1 --2- 172.21.15.33:0/2696135611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b4070010 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.147+0000 7f57bbca4700 1 --2- 172.21.15.33:0/2696135611 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b4070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.147+0000 7f57bbca4700 1 -- 172.21.15.33:0/2696135611 >> 172.21.15.33:0/2696135611 conn(0x7f57b40c8360 msgr2=0x7f57b40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:56.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.148+0000 7f57bbca4700 1 -- 172.21.15.33:0/2696135611 shutdown_connections 2024-05-08T23:36:56.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.150+0000 7f57bbca4700 1 -- 172.21.15.33:0/2696135611 wait complete. 2024-05-08T23:36:56.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.151+0000 7f57bbca4700 1 Processor -- start 2024-05-08T23:36:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.151+0000 7f57bbca4700 1 -- start start 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b408b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 0x7f57b408bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4200590 con 0x7f57b40c58c0 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4157280 con 0x7f57b41670c0 2024-05-08T23:36:56.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.152+0000 7f57bbca4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b4157720 con 0x7f57b40c9050 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.153+0000 7f57ba241700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 msgr2=0x7f57b408bed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.153+0000 7f57ba241700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 0x7f57b408bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.153+0000 7f57b9a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.153+0000 7f57b9a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54368/0 (socket says 172.21.15.33:54368) 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.153+0000 7f57b9a40700 1 -- 172.21.15.33:0/1020279265 learned_addr learned my addr 172.21.15.33:0/1020279265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 -- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 msgr2=0x7f57b408b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b408b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 -- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 msgr2=0x7f57b408bed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:36:56.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 0x7f57b408bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 -- 172.21.15.33:0/1020279265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0009550 con 0x7f57b40c58c0 2024-05-08T23:36:56.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.154+0000 7f57b9a40700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009c40 tx=0x7f57b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.161+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000b660 con 0x7f57b40c58c0 2024-05-08T23:36:56.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.161+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000b7c0 con 0x7f57b40c58c0 2024-05-08T23:36:56.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.161+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000bb70 con 0x7f57b40c58c0 2024-05-08T23:36:56.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.161+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b4176320 con 0x7f57b40c58c0 2024-05-08T23:36:56.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.161+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b4157c70 con 0x7f57b40c58c0 2024-05-08T23:36:56.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.164+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f57b0033070 con 0x7f57b40c58c0 2024-05-08T23:36:56.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.164+0000 7f57aa7fc700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 0x7f579806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:36:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.165+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f57b009bc60 con 0x7f57b40c58c0 2024-05-08T23:36:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.165+0000 7f57b923f700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 0x7f579806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:36:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.165+0000 7f57b923f700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 0x7f579806d1d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57b40c4470 tx=0x7f57a4006cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:36:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.168+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c0052f0 con 0x7f57b40c58c0 2024-05-08T23:36:56.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.176+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f57b0066de0 con 0x7f57b40c58c0 2024-05-08T23:36:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.940+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f579c005b20 con 0x7f57b40c58c0 2024-05-08T23:36:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.947+0000 7f57aa7fc700 1 -- 172.21.15.33:0/1020279265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f57b003b030 con 0x7f57b40c58c0 2024-05-08T23:36:56.946 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.953+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 msgr2=0x7f579806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.953+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 0x7f579806d1d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57b40c4470 tx=0x7f57a4006cd0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.953+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 msgr2=0x7f57b416a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.953+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009c40 tx=0x7f57b0004750 comp rx=0 tx=0).stop 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 shutdown_connections 2024-05-08T23:36:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f579806ad10 0x7f579806d1d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b41670c0 0x7f57b408bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40c58c0 0x7f57b416a870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 --2- 172.21.15.33:0/1020279265 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f57b40c9050 0x7f57b408b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 >> 172.21.15.33:0/1020279265 conn(0x7f57b40c8360 msgr2=0x7f57b406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 shutdown_connections 2024-05-08T23:36:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:36:56.954+0000 7f57bbca4700 1 -- 172.21.15.33:0/1020279265 wait complete. 2024-05-08T23:36:59.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:00.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.494+0000 7ff2e52dc700 1 Processor -- start 2024-05-08T23:37:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.494+0000 7ff2e52dc700 1 -- start start 2024-05-08T23:37:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.496+0000 7ff2e52dc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.496+0000 7ff2e52dc700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e0158ee0 con 0x7ff2e00722c0 2024-05-08T23:37:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.496+0000 7ff2e52dc700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e006c730 con 0x7ff2e00c70d0 2024-05-08T23:37:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.496+0000 7ff2e52dc700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00c2e60 con 0x7ff2e00c94e0 2024-05-08T23:37:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.496+0000 7ff2de7fc700 1 -- >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2e0161c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.497+0000 7ff2deffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2e00722c0 0x7ff2e00c69c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44272/0 (socket says 172.21.15.33:44272) 2024-05-08T23:37:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.497+0000 7ff2deffd700 1 -- 172.21.15.33:0/4116175773 learned_addr learned my addr 172.21.15.33:0/4116175773 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.504+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:00.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.515+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:37:00.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.515+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:00.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.515+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:00.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.515+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.697+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2e0161c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:00.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.718+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:00.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.718+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:37:00.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.718+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:00.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.718+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:00.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:00.718+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:37:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.100+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2e0161c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.122+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.123+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:37:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.123+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.123+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.123+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:37:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.901+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2e0161c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.927+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.928+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:37:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.928+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.928+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:01.928+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 msgr2=0x7ff2e0162380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2e00c94e0 0x7ff2e0162380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2e0161c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.33:6789/0 conn(0x7ff2e00722c0 legacy=0x7ff2e00c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00c2e60 con 0x7ff2cc006250 2024-05-08T23:37:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2cc008dc0 con 0x7ff2cc003780 2024-05-08T23:37:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2deffd700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 msgr2=0x7ff2cc005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.497+0000 7ff2deffd700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.498+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.498+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:37:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.498+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.498+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.498+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:03.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.698+0000 7ff2deffd700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 msgr2=0x7ff2cc005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:03.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.698+0000 7ff2deffd700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:37:03.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.702+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:03.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.702+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:37:03.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.702+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:03.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.702+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:03.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:03.702+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:37:04.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.101+0000 7ff2deffd700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 msgr2=0x7ff2cc005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:04.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.101+0000 7ff2deffd700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:37:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.113+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.114+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:37:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.114+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.114+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.114+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:37:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.906+0000 7ff2deffd700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 msgr2=0x7ff2cc005c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.906+0000 7ff2deffd700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:37:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.916+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.916+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:37:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.916+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:37:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.916+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:37:04.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:04.916+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:37:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 msgr2=0x7ff2cc005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc003780 0x7ff2cc005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 msgr2=0x7ff2cc008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.33:3300/0 conn(0x7ff2cc006250 0x7ff2cc008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc00c900 0x7ff2cc00ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e006ca10 con 0x7ff2e00722c0 2024-05-08T23:37:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e006c730 con 0x7ff2e00c70d0 2024-05-08T23:37:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.497+0000 7ff2dcff9700 1 -- 172.21.15.33:0/4116175773 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00c2e60 con 0x7ff2cc00c900 2024-05-08T23:37:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.498+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 reap_dead start 2024-05-08T23:37:06.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.498+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2cc0114b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:06.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.500+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc00c900 msgr2=0x7ff2cc00ed90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:06.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.500+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc00c900 0x7ff2cc00ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.501+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422011235 0 0) 0x7ff2e006ca10 con 0x7ff2e00722c0 2024-05-08T23:37:06.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.501+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c80035b0 con 0x7ff2e00722c0 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 321047716 0 0) 0x7ff2c80035b0 con 0x7ff2e00722c0 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc00c900 msgr2=0x7ff2cc00ed90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 --2- 172.21.15.33:0/4116175773 >> v2:172.21.15.42:3300/0 conn(0x7ff2cc00c900 0x7ff2cc00ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.42:6789/0 conn(0x7ff2e00c70d0 legacy=0x7ff2cc0114b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc014ce0 con 0x7ff2e00722c0 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7ff2d00017f0 con 0x7ff2e00722c0 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 >> v1:172.21.15.33:6789/0 conn(0x7ff2e00722c0 legacy=0x7ff2cc00b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8003a00 0x7ff2c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c8004850 0x7ff2c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 0x7ff2c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00c2e60 con 0x7ff2c8005330 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c230 con 0x7ff2c8004850 2024-05-08T23:37:06.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.504+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c3f0 con 0x7ff2c8003a00 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.505+0000 7ff2de7fc700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c8004850 msgr2=0x7ff2c8004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.505+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c8004850 0x7ff2c8004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.505+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 0x7ff2c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.505+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8003a00 msgr2=0x7ff2c8003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8003a00 0x7ff2c8003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c8004850 msgr2=0x7ff2c8004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c8004850 0x7ff2c8004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc014ce0 con 0x7ff2c8005330 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 -- 172.21.15.33:0/4116175773 reap_dead start 2024-05-08T23:37:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.506+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 0x7ff2c800ba10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2e006fd60 tx=0x7ff2c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f730a8415a2df720 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.508+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004220 con 0x7ff2c8005330 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.508+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400c070 con 0x7ff2c8005330 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.508+0000 7ff2ddffb700 1 -- 172.21.15.33:0/4116175773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40036a0 con 0x7ff2c8005330 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 -- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 msgr2=0x7ff2c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 0x7ff2c800ba10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2e006fd60 tx=0x7ff2c8005880 comp rx=0 tx=0).stop 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 -- 172.21.15.33:0/4116175773 shutdown_connections 2024-05-08T23:37:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/4116175773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2c8005330 0x7ff2c800ba10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 -- 172.21.15.33:0/4116175773 >> 172.21.15.33:0/4116175773 conn(0x7ff2e00c89c0 msgr2=0x7ff2e00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.509+0000 7ff2e52dc700 1 -- 172.21.15.33:0/4116175773 shutdown_connections 2024-05-08T23:37:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.510+0000 7ff2e52dc700 1 -- 172.21.15.33:0/4116175773 wait complete. 2024-05-08T23:37:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.512+0000 7ff2e52dc700 1 Processor -- start 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.513+0000 7ff2e52dc700 1 -- start start 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8005330 0x7ff2e015b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 0x7ff2e00c3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e01f4eb0 con 0x7ff2e015c570 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e01f4020 con 0x7ff2e00c59d0 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.514+0000 7ff2e52dc700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00c5e50 con 0x7ff2c8005330 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.515+0000 7ff2de7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.515+0000 7ff2de7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58344/0 (socket says 172.21.15.33:58344) 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.515+0000 7ff2de7fc700 1 -- 172.21.15.33:0/1841244772 learned_addr learned my addr 172.21.15.33:0/1841244772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.515+0000 7ff2deffd700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8005330 0x7ff2e015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.516+0000 7ff2df7fe700 1 -- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 msgr2=0x7ff2e00c3280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.516+0000 7ff2df7fe700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 0x7ff2e00c3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.516+0000 7ff2de7fc700 1 -- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8005330 msgr2=0x7ff2e015b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.516+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8005330 0x7ff2e015b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.517+0000 7ff2de7fc700 1 -- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 msgr2=0x7ff2e00c3280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.517+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 0x7ff2e00c3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.517+0000 7ff2de7fc700 1 -- 172.21.15.33:0/1841244772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e01e4ab0 con 0x7ff2e015c570 2024-05-08T23:37:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.517+0000 7ff2de7fc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800db80 tx=0x7ff2d800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.518+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800cdd0 con 0x7ff2e015c570 2024-05-08T23:37:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.518+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d8013070 con 0x7ff2e015c570 2024-05-08T23:37:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.518+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80196b0 con 0x7ff2e015c570 2024-05-08T23:37:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.520+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01f42a0 con 0x7ff2e015c570 2024-05-08T23:37:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.521+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e01f4860 con 0x7ff2e015c570 2024-05-08T23:37:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.522+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e006bf40 con 0x7ff2e015c570 2024-05-08T23:37:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.525+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff2d800a400 con 0x7ff2e015c570 2024-05-08T23:37:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.526+0000 7ff2c77fe700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 0x7ff2c806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.526+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff2d808a3c0 con 0x7ff2e015c570 2024-05-08T23:37:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.526+0000 7ff2deffd700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 0x7ff2c806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.526+0000 7ff2deffd700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 0x7ff2c806e910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0074aa0 tx=0x7ff2d0007c70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:06.555+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff2d8055540 con 0x7ff2e015c570 2024-05-08T23:37:07.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.327+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff2e006ca10 con 0x7ff2e015c570 2024-05-08T23:37:07.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.331+0000 7ff2c77fe700 1 -- 172.21.15.33:0/1841244772 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7ff2d8058b60 con 0x7ff2e015c570 2024-05-08T23:37:07.348 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":42,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.359+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 msgr2=0x7ff2c806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.359+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 0x7ff2c806e910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0074aa0 tx=0x7ff2d0007c70 comp rx=0 tx=0).stop 2024-05-08T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.359+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 msgr2=0x7ff2e00c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.359+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800db80 tx=0x7ff2d800de90 comp rx=0 tx=0).stop 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 shutdown_connections 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2c806c450 0x7ff2c806e910 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2e00c59d0 0x7ff2e00c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2c8005330 0x7ff2e015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 --2- 172.21.15.33:0/1841244772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e015c570 0x7ff2e00c5490 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.361+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 >> 172.21.15.33:0/1841244772 conn(0x7ff2e00c89c0 msgr2=0x7ff2e00ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:07.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.370+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 shutdown_connections 2024-05-08T23:37:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.372+0000 7ff2e52dc700 1 -- 172.21.15.33:0/1841244772 wait complete. 2024-05-08T23:37:07.395 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:37:07.395 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-08T23:37:07.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:37:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.751+0000 7f7fe1a99700 1 Processor -- start 2024-05-08T23:37:07.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.758+0000 7f7fe1a99700 1 -- start start 2024-05-08T23:37:07.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.759+0000 7f7fe1a99700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7fdc1556e0 0x7f7fdc06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.760+0000 7f7fe1a99700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0c5dc0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.760+0000 7f7fe1a99700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc162c90 con 0x7f7fdc0c7ae0 2024-05-08T23:37:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.760+0000 7f7fe1a99700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc06b180 con 0x7f7fdc1556e0 2024-05-08T23:37:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fdb7fe700 1 -- >> v2:172.21.15.42:3300/0 conn(0x7f7fdc1556e0 msgr2=0x7f7fdc06b610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fdb7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7fdc1556e0 0x7f7fdc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fdaffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7fdc0c9050 0x7f7fdc070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56972/0 (socket says 172.21.15.33:56972) 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fdaffd700 1 -- 172.21.15.33:0/1243648743 learned_addr learned my addr 172.21.15.33:0/1243648743 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fdbfff700 1 -- 172.21.15.33:0/1243648743 >> v1:172.21.15.42:6789/0 conn(0x7f7fdc0c7ae0 legacy=0x7f7fdc169070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.761+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533260249 0 0) 0x7f7fdc0c5dc0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc40035b0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165210375 0 0) 0x7f7fc40035b0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 >> v2:172.21.15.42:3300/0 conn(0x7f7fdc1556e0 msgr2=0x7f7fdc06b610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 --2- 172.21.15.33:0/1243648743 >> v2:172.21.15.42:3300/0 conn(0x7f7fdc1556e0 0x7f7fdc06b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 >> v1:172.21.15.42:6789/0 conn(0x7f7fdc0c7ae0 legacy=0x7f7fdc169070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc1629b0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f7fd00015a0 con 0x7f7fdc0c9050 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 >> v1:172.21.15.33:6789/0 conn(0x7f7fdc0c9050 legacy=0x7f7fdc070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 0x7f7fc4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fc4004850 0x7f7fc4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fc4005330 0x7f7fc400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc06b180 con 0x7f7fc4003a00 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc400c220 con 0x7f7fc4004850 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc400c3e0 con 0x7f7fc4005330 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.762+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 0x7f7fc4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdaffd700 1 -- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fc4004850 msgr2=0x7f7fc4004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdaffd700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fc4004850 0x7f7fc4004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fc4005330 msgr2=0x7f7fc400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fc4005330 0x7f7fc400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fc4004850 msgr2=0x7f7fc4004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fc4004850 0x7f7fc4004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.763+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1243648743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc1629b0 con 0x7f7fc4003a00 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.765+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 0x7f7fc4003e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0039c0 tx=0x7f7fc4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877d53144bbf964d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.768+0000 7f7fdbfff700 1 -- 172.21.15.33:0/1243648743 reap_dead start 2024-05-08T23:37:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.769+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc01cd70 con 0x7f7fc4003a00 2024-05-08T23:37:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.769+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc003a40 con 0x7f7fc4003a00 2024-05-08T23:37:07.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.769+0000 7f7fda7fc700 1 -- 172.21.15.33:0/1243648743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc003dd0 con 0x7f7fc4003a00 2024-05-08T23:37:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.770+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 msgr2=0x7f7fc4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.770+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 0x7f7fc4003e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0039c0 tx=0x7f7fc4005880 comp rx=0 tx=0).stop 2024-05-08T23:37:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.770+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1243648743 shutdown_connections 2024-05-08T23:37:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.770+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1243648743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fc4003a00 0x7f7fc4003e00 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0039c0 tx=0x7f7fc4005880 comp rx=0 tx=0).stop 2024-05-08T23:37:07.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.770+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1243648743 >> 172.21.15.33:0/1243648743 conn(0x7f7fdc0c8360 msgr2=0x7f7fdc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:07.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.772+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1243648743 shutdown_connections 2024-05-08T23:37:07.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.779+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1243648743 wait complete. 2024-05-08T23:37:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.781+0000 7f7fe1a99700 1 Processor -- start 2024-05-08T23:37:07.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.781+0000 7f7fe1a99700 1 -- start start 2024-05-08T23:37:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 0x7f7fdc161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc160150 0x7f7fdc160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fdc1462e0 0x7f7fdc146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc146cb0 con 0x7f7fdc160150 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc1615a0 con 0x7f7fdc1462e0 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fe1a99700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc14c260 con 0x7f7fdc15fd70 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdbfff700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fdc1462e0 msgr2=0x7f7fdc146770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdb7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 0x7f7fdc161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fdc1462e0 0x7f7fdc146770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdb7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 0x7f7fdc161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:57586/0 (socket says 172.21.15.33:57586) 2024-05-08T23:37:07.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1699850247 learned_addr learned my addr 172.21.15.33:0/1699850247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fdc1462e0 msgr2=0x7f7fdc146770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.783+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fdc1462e0 0x7f7fdc146770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.784+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc160150 msgr2=0x7f7fdc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.784+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc160150 0x7f7fdc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.784+0000 7f7fdb7fe700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc15bc10 con 0x7f7fdc15fd70 2024-05-08T23:37:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.784+0000 7f7fdb7fe700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 0x7f7fdc161840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc01cff0 tx=0x7f7fcc005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.789+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc005610 con 0x7f7fdc15fd70 2024-05-08T23:37:07.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.789+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc02b070 con 0x7f7fdc15fd70 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.789+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc031470 con 0x7f7fdc15fd70 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.791+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc15bef0 con 0x7f7fdc15fd70 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.791+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc1612c0 con 0x7f7fdc15fd70 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.792+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 0x7f7fdc14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.792+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc14c260 con 0x7f7fdc14c710 2024-05-08T23:37:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.792+0000 7f7fdaffd700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 0x7f7fdc14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:07.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.793+0000 7f7fdaffd700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fdc1626a0 con 0x7f7fdc14c710 2024-05-08T23:37:07.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.793+0000 7f7fdaffd700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 0x7f7fdc14cb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0cadf0 tx=0x7f7fd000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0004c80 con 0x7f7fdc14c710 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7fd0007e20 con 0x7f7fdc14c710 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 msgr2=0x7f7fdc14cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 0x7f7fdc14cb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0cadf0 tx=0x7f7fd000cd80 comp rx=0 tx=0).stop 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 msgr2=0x7f7fdc14cb30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:07.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.795+0000 7f7fc3fff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc14c710 0x7f7fdc14cb30 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0cadf0 tx=0x7f7fd000cd80 comp rx=0 tx=0).stop 2024-05-08T23:37:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.806+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7fcc026400 con 0x7f7fdc15fd70 2024-05-08T23:37:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.806+0000 7f7fc3fff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7fc806aa10 0x7f7fc806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.807+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7fcc008050 con 0x7f7fdc15fd70 2024-05-08T23:37:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.807+0000 7f7fdaffd700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7fc806aa10 0x7f7fc806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:07.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.808+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 0x7f7fdc1ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.808+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc074a20 con 0x7f7fdc1eee70 2024-05-08T23:37:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.808+0000 7f7fdbfff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 0x7f7fdc1ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.809+0000 7f7fdbfff700 1 -- 172.21.15.33:0/1699850247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7fdc158f20 con 0x7f7fdc1eee70 2024-05-08T23:37:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.809+0000 7f7fdbfff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 0x7f7fdc1ef250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc06cf50 tx=0x7f7fd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fdaffd700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7fc806aa10 0x7f7fc806ced0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00016d0 tx=0x7f7fd0009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:07.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400cdf0 con 0x7f7fdc1eee70 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5101 (secure 0 0 0) 0x7f7fd4014420 con 0x7f7fdc1eee70 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 msgr2=0x7f7fdc1ef250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 0x7f7fdc1ef250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc06cf50 tx=0x7f7fd4009e30 comp rx=0 tx=0).stop 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 msgr2=0x7f7fdc1ef250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.810+0000 7f7fc3fff700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fdc1eee70 0x7f7fdc1ef250 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc06cf50 tx=0x7f7fd4009e30 comp rx=0 tx=0).stop 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:37:07.808 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 42, 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:07.809 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:07.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:07.812 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.818 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 7 2024-05-08T23:37:07.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:07.820 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:07.821 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7fc806aa10 msgr2=0x7f7fc806ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7fc806aa10 0x7f7fc806ced0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00016d0 tx=0x7f7fd0009000 comp rx=0 tx=0).stop 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 msgr2=0x7f7fdc161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 --2- 172.21.15.33:0/1699850247 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7fdc15fd70 0x7f7fdc161840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc01cff0 tx=0x7f7fcc005aa0 comp rx=0 tx=0).stop 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fdbfff700 1 -- 172.21.15.33:0/1699850247 reap_dead start 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 shutdown_connections 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 >> 172.21.15.33:0/1699850247 conn(0x7f7fdc0c8360 msgr2=0x7f7fdc0ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 shutdown_connections 2024-05-08T23:37:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:07.811+0000 7f7fe1a99700 1 -- 172.21.15.33:0/1699850247 wait complete. 2024-05-08T23:37:07.837 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:37:07.837 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:37:07.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:37:08.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.118+0000 7f066e5a0700 1 Processor -- start 2024-05-08T23:37:08.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.122+0000 7f066e5a0700 1 -- start start 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.123+0000 7f066e5a0700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06681545c0 0x7f06680c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.123+0000 7f066e5a0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 0x7f06680c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.123+0000 7f066e5a0700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06680c32d0 con 0x7f06680c87e0 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.123+0000 7f066e5a0700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0668070310 con 0x7f06681545c0 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.123+0000 7f066e5a0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066815aec0 con 0x7f06680c94a0 2024-05-08T23:37:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.124+0000 7f066cb3d700 1 -- >> v1:172.21.15.42:6789/0 conn(0x7f06680c87e0 legacy=0x7f06680cdd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.42:6789/0 2024-05-08T23:37:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.126+0000 7f06677fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 0x7f06680c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.126+0000 7f06677fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 0x7f06680c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:57594/0 (socket says 172.21.15.33:57594) 2024-05-08T23:37:08.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.126+0000 7f06677fe700 1 -- 172.21.15.33:0/2488524923 learned_addr learned my addr 172.21.15.33:0/2488524923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:08.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.130+0000 7f0667fff700 1 --2- 172.21.15.33:0/2488524923 >> v2:172.21.15.33:3300/0 conn(0x7f06681545c0 0x7f06680c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.131+0000 7f0667fff700 1 -- 172.21.15.33:0/2488524923 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 msgr2=0x7f06680c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.131+0000 7f0667fff700 1 --2- 172.21.15.33:0/2488524923 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 0x7f06680c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.131+0000 7f0667fff700 1 -- 172.21.15.33:0/2488524923 >> v1:172.21.15.42:6789/0 conn(0x7f06680c87e0 legacy=0x7f06680cdd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.131+0000 7f0667fff700 1 -- 172.21.15.33:0/2488524923 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066815b1a0 con 0x7f06681545c0 2024-05-08T23:37:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.131+0000 7f0667fff700 1 --2- 172.21.15.33:0/2488524923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f06680c4c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0658009840 tx=0x7f0658009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95e7306ea2da6f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.132+0000 7f0666ffd700 1 -- 172.21.15.33:0/2488524923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658004230 con 0x7f06681545c0 2024-05-08T23:37:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.132+0000 7f0666ffd700 1 -- 172.21.15.33:0/2488524923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0658004390 con 0x7f06681545c0 2024-05-08T23:37:08.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.132+0000 7f0666ffd700 1 -- 172.21.15.33:0/2488524923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580036a0 con 0x7f06681545c0 2024-05-08T23:37:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.133+0000 7f066e5a0700 1 -- 172.21.15.33:0/2488524923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 msgr2=0x7f06680c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.133+0000 7f066e5a0700 1 --2- 172.21.15.33:0/2488524923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f06680c4c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0658009840 tx=0x7f0658009b50 comp rx=0 tx=0).stop 2024-05-08T23:37:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 -- 172.21.15.33:0/2488524923 shutdown_connections 2024-05-08T23:37:08.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 --2- 172.21.15.33:0/2488524923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f06680c4c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 --2- 172.21.15.33:0/2488524923 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c94a0 0x7f06680c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 -- 172.21.15.33:0/2488524923 >> 172.21.15.33:0/2488524923 conn(0x7f066806a990 msgr2=0x7f06680666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 -- 172.21.15.33:0/2488524923 shutdown_connections 2024-05-08T23:37:08.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.137+0000 7f066e5a0700 1 -- 172.21.15.33:0/2488524923 wait complete. 2024-05-08T23:37:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.139+0000 7f066e5a0700 1 Processor -- start 2024-05-08T23:37:08.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.140+0000 7f066e5a0700 1 -- start start 2024-05-08T23:37:08.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 0x7f06680c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680c94a0 0x7f06681580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f066813f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681490e0 con 0x7f06681545c0 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066814a840 con 0x7f06680c94a0 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f066e5a0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0668149470 con 0x7f06680c87e0 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f06677fe700 1 -- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680c94a0 msgr2=0x7f06681580a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:3300/0 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f06677fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680c94a0 0x7f06681580a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f0667fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 0x7f06680c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f0667fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 0x7f06680c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:57608/0 (socket says 172.21.15.33:57608) 2024-05-08T23:37:08.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.141+0000 7f0667fff700 1 -- 172.21.15.33:0/3554752765 learned_addr learned my addr 172.21.15.33:0/3554752765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:08.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.145+0000 7f066cb3d700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f066813f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.146+0000 7f0667fff700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680c94a0 msgr2=0x7f06681580a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.146+0000 7f0667fff700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680c94a0 0x7f06681580a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.146+0000 7f0667fff700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 msgr2=0x7f066813f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.146+0000 7f0667fff700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06681545c0 0x7f066813f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:08.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.146+0000 7f0667fff700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580094a0 con 0x7f06680c87e0 2024-05-08T23:37:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f0667fff700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 0x7f06680c7830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658003e90 con 0x7f06680c87e0 2024-05-08T23:37:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065801f070 con 0x7f06680c87e0 2024-05-08T23:37:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658011850 con 0x7f06680c87e0 2024-05-08T23:37:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680c69b0 con 0x7f06680c87e0 2024-05-08T23:37:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.147+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06680c6c50 con 0x7f06680c87e0 2024-05-08T23:37:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.153+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f06580119b0 con 0x7f06680c87e0 2024-05-08T23:37:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.154+0000 7f0664ff9700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f065406ac90 0x7f065406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.154+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0658021070 con 0x7f06680c87e0 2024-05-08T23:37:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.154+0000 7f06677fe700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f065406ac90 0x7f065406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.155+0000 7f066e5a0700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 0x7f0648004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.155+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0648007f20 con 0x7f0648002a40 2024-05-08T23:37:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.155+0000 7f06677fe700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f065406ac90 0x7f065406d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0668154320 tx=0x7f065c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.157+0000 7f066cb3d700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 0x7f0648004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.173+0000 7f066cb3d700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f06480080e0 con 0x7f0648002a40 2024-05-08T23:37:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.178+0000 7f066cb3d700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 0x7f0648004f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f066000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.188+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660011070 con 0x7f0648002a40 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.189+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f066000be40 con 0x7f0648002a40 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.189+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 msgr2=0x7f0648004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.189+0000 7f0664ff9700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 0x7f0648004f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f066000b690 comp rx=0 tx=0).stop 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.189+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 msgr2=0x7f0648004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:08.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.189+0000 7f0664ff9700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648002a40 0x7f0648004f00 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f066000b690 comp rx=0 tx=0).stop 2024-05-08T23:37:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.200+0000 7f066e5a0700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 0x7f0648007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:08.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.200+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064800a3e0 con 0x7f0648006fc0 2024-05-08T23:37:08.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.201+0000 7f066cb3d700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 0x7f0648007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.212+0000 7f066cb3d700 1 -- 172.21.15.33:0/3554752765 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f064800a5a0 con 0x7f0648006fc0 2024-05-08T23:37:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.212+0000 7f066cb3d700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 0x7f0648007450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f0660003c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660021070 con 0x7f0648006fc0 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7f0660016750 con 0x7f0648006fc0 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 msgr2=0x7f0648007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 0x7f0648007450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f0660003c70 comp rx=0 tx=0).stop 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 msgr2=0x7f0648007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:08.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.214+0000 7f0664ff9700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0648006fc0 0x7f0648007450 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f06680c3070 tx=0x7f0660003c70 comp rx=0 tx=0).stop 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 42, 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:08.213 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:08.225 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.216+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f065406ac90 msgr2=0x7f065406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.216+0000 7f066e5a0700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f065406ac90 0x7f065406d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0668154320 tx=0x7f065c008040 comp rx=0 tx=0).stop 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.216+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 msgr2=0x7f06680c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.216+0000 7f066e5a0700 1 --2- 172.21.15.33:0/3554752765 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f06680c87e0 0x7f06680c7830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0658000c00 tx=0x7f0658004750 comp rx=0 tx=0).stop 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.216+0000 7f066cb3d700 1 -- 172.21.15.33:0/3554752765 reap_dead start 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.217+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 shutdown_connections 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.217+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 >> 172.21.15.33:0/3554752765 conn(0x7f066806a990 msgr2=0x7f06681536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.219+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 shutdown_connections 2024-05-08T23:37:08.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:08.219+0000 7f066e5a0700 1 -- 172.21.15.33:0/3554752765 wait complete. 2024-05-08T23:37:08.249 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:37:08.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:37:19.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:37:19.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:37:28.250 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T23:37:28.250 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-05-08T23:37:28.250 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T23:37:28.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T23:37:28.263 INFO:tasks.ceph.mon.b:Started 2024-05-08T23:37:28.263 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:37:28.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:28.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.809+0000 7f18266a2700 1 Processor -- start 2024-05-08T23:37:28.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.812+0000 7f18266a2700 1 -- start start 2024-05-08T23:37:28.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.813+0000 7f18266a2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1820154f90 0x7f182006ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.813+0000 7f18266a2700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182006c260 con 0x7f18200c9350 2024-05-08T23:37:28.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.813+0000 7f18266a2700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182006c6b0 con 0x7f1820162470 2024-05-08T23:37:28.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.813+0000 7f18266a2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182006cc70 con 0x7f1820154f90 2024-05-08T23:37:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.827+0000 7f1824c3f700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f1820162470 0x7f1820169010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:48694/0 (socket says 172.21.15.33:48694) 2024-05-08T23:37:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.827+0000 7f1824c3f700 1 -- 172.21.15.33:0/1916816706 learned_addr learned my addr 172.21.15.33:0/1916816706 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929852842 0 0) 0x7f182006c260 con 0x7f18200c9350 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18080035b0 con 0x7f18200c9350 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 726666364 0 0) 0x7f18080035b0 con 0x7f18200c9350 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1820154f90 msgr2=0x7f182006ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1820154f90 0x7f182006ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 >> v1:172.21.15.42:6789/0 conn(0x7f1820162470 legacy=0x7f1820169010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182006c990 con 0x7f18200c9350 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f1814001670 con 0x7f18200c9350 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 >> v1:172.21.15.33:6789/0 conn(0x7f18200c9350 legacy=0x7f1820161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.840+0000 7f181effd700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1808003a30 0x7f1808003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181effd700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f1808004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181effd700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1808005280 0x7f180800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182006cc70 con 0x7f1808003a30 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180801d1b0 con 0x7f1808005280 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180801d370 con 0x7f1808004850 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f1808004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 -- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1808005280 msgr2=0x7f180800ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1808005280 0x7f180800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 -- 172.21.15.33:0/1916816706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1808003a30 msgr2=0x7f1808003e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1808003a30 0x7f1808003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.841+0000 7f181f7fe700 1 -- 172.21.15.33:0/1916816706 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080035b0 con 0x7f1808004850 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.842+0000 7f181f7fe700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f1808004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f181400c160 tx=0x7f1808005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97a3c192a193751 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.843+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400cd00 con 0x7f1808004850 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.843+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181400ce60 con 0x7f1808004850 2024-05-08T23:37:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.843+0000 7f181effd700 1 -- 172.21.15.33:0/1916816706 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814006a80 con 0x7f1808004850 2024-05-08T23:37:28.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.851+0000 7f18266a2700 1 -- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 msgr2=0x7f1808004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:28.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.851+0000 7f18266a2700 1 --2- 172.21.15.33:0/1916816706 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f1808004cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f181400c160 tx=0x7f1808005880 comp rx=0 tx=0).stop 2024-05-08T23:37:28.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.852+0000 7f1824c3f700 1 -- 172.21.15.33:0/1916816706 reap_dead start 2024-05-08T23:37:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.853+0000 7f18266a2700 1 -- 172.21.15.33:0/1916816706 shutdown_connections 2024-05-08T23:37:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.853+0000 7f18266a2700 1 -- 172.21.15.33:0/1916816706 >> 172.21.15.33:0/1916816706 conn(0x7f18200ccc50 msgr2=0x7f18200cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:28.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.853+0000 7f18266a2700 1 -- 172.21.15.33:0/1916816706 shutdown_connections 2024-05-08T23:37:28.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.853+0000 7f18266a2700 1 -- 172.21.15.33:0/1916816706 wait complete. 2024-05-08T23:37:28.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.855+0000 7f18266a2700 1 Processor -- start 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.856+0000 7f18266a2700 1 -- start start 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f182014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18200cc5a0 0x7f182015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820158ff0 con 0x7f1820152cc0 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18200c7eb0 con 0x7f18200cc5a0 2024-05-08T23:37:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.858+0000 7f18266a2700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200c8190 con 0x7f1808004850 2024-05-08T23:37:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52412/0 (socket says 172.21.15.33:52412) 2024-05-08T23:37:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 -- 172.21.15.33:0/316706111 learned_addr learned my addr 172.21.15.33:0/316706111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 -- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 msgr2=0x7f182014aff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f182014aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 -- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18200cc5a0 msgr2=0x7f182015aed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18200cc5a0 0x7f182015aed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.859+0000 7f1824c3f700 1 -- 172.21.15.33:0/316706111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820160ab0 con 0x7f1820152cc0 2024-05-08T23:37:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.860+0000 7f1824c3f700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f182015e690 tx=0x7f181800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.862+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800a930 con 0x7f1820152cc0 2024-05-08T23:37:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.862+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181800aa90 con 0x7f1820152cc0 2024-05-08T23:37:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.862+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181800ae00 con 0x7f1820152cc0 2024-05-08T23:37:28.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.863+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1820160d90 con 0x7f1820152cc0 2024-05-08T23:37:28.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.863+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182014b530 con 0x7f1820152cc0 2024-05-08T23:37:28.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.865+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1820162b30 con 0x7f1820152cc0 2024-05-08T23:37:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.876+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f181801b3b0 con 0x7f1820152cc0 2024-05-08T23:37:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.877+0000 7f181cff9700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 0x7f180c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.877+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f18180891b0 con 0x7f1820152cc0 2024-05-08T23:37:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.877+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1818089720 con 0x7f1820152cc0 2024-05-08T23:37:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.880+0000 7f181ffff700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 0x7f180c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:28.880+0000 7f181ffff700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 0x7f180c06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f182006da10 tx=0x7f1810006cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:29.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.678+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1820075460 con 0x7f1820152cc0 2024-05-08T23:37:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.684+0000 7f181cff9700 1 -- 172.21.15.33:0/316706111 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f1818017610 con 0x7f1820152cc0 2024-05-08T23:37:29.684 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":42,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.718+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 msgr2=0x7f180c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.718+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 0x7f180c06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f182006da10 tx=0x7f1810006cb0 comp rx=0 tx=0).stop 2024-05-08T23:37:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.718+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 msgr2=0x7f18201ef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:29.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.718+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f182015e690 tx=0x7f181800ce00 comp rx=0 tx=0).stop 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 shutdown_connections 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f180c06ac90 0x7f180c06d150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1820152cc0 0x7f18201ef520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1808004850 0x7f182014aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 --2- 172.21.15.33:0/316706111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18200cc5a0 0x7f182015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.719+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 >> 172.21.15.33:0/316706111 conn(0x7f18200ccc50 msgr2=0x7f182006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.723+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 shutdown_connections 2024-05-08T23:37:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:29.729+0000 7f18266a2700 1 -- 172.21.15.33:0/316706111 wait complete. 2024-05-08T23:37:32.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.169+0000 7f11146aa700 1 Processor -- start 2024-05-08T23:37:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.169+0000 7f11146aa700 1 -- start start 2024-05-08T23:37:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.171+0000 7f11146aa700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f110c0b3bd0 0x7f110c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.171+0000 7f11146aa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.171+0000 7f11146aa700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c15b610 con 0x7f110c152160 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.171+0000 7f11146aa700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c15c5a0 con 0x7f110c0b3bd0 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.171+0000 7f11146aa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0708e0 con 0x7f110c158020 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1112446700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f110c152160 0x7f110c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59270/0 (socket says 172.21.15.33:59270) 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1112446700 1 -- 172.21.15.33:0/3249265053 learned_addr learned my addr 172.21.15.33:0/3249265053 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1111c45700 1 --2- 172.21.15.33:0/3249265053 >> v2:172.21.15.33:3300/0 conn(0x7f110c0b3bd0 0x7f110c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1111c45700 1 -- 172.21.15.33:0/3249265053 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 msgr2=0x7f110c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1111c45700 1 --2- 172.21.15.33:0/3249265053 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1111c45700 1 -- 172.21.15.33:0/3249265053 >> v1:172.21.15.33:6789/0 conn(0x7f110c152160 legacy=0x7f110c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.173+0000 7f1111c45700 1 -- 172.21.15.33:0/3249265053 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0c8910 con 0x7f110c0b3bd0 2024-05-08T23:37:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.174+0000 7f1111c45700 1 --2- 172.21.15.33:0/3249265053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c0b3bd0 0x7f110c15c060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1108009840 tx=0x7f1108009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f7b652d3e93c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.186+0000 7f1111444700 1 -- 172.21.15.33:0/3249265053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108011830 con 0x7f110c0b3bd0 2024-05-08T23:37:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.186+0000 7f1111444700 1 -- 172.21.15.33:0/3249265053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1108011990 con 0x7f110c0b3bd0 2024-05-08T23:37:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.186+0000 7f1111444700 1 -- 172.21.15.33:0/3249265053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108011d20 con 0x7f110c0b3bd0 2024-05-08T23:37:33.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.187+0000 7f11146aa700 1 -- 172.21.15.33:0/3249265053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c0b3bd0 msgr2=0x7f110c15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:33.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.187+0000 7f11146aa700 1 --2- 172.21.15.33:0/3249265053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c0b3bd0 0x7f110c15c060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1108009840 tx=0x7f1108009b50 comp rx=0 tx=0).stop 2024-05-08T23:37:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.193+0000 7f11146aa700 1 -- 172.21.15.33:0/3249265053 shutdown_connections 2024-05-08T23:37:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.194+0000 7f11146aa700 1 --2- 172.21.15.33:0/3249265053 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.194+0000 7f11146aa700 1 --2- 172.21.15.33:0/3249265053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c0b3bd0 0x7f110c15c060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.194+0000 7f11146aa700 1 -- 172.21.15.33:0/3249265053 >> 172.21.15.33:0/3249265053 conn(0x7f110c072230 msgr2=0x7f110c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.195+0000 7f11146aa700 1 -- 172.21.15.33:0/3249265053 shutdown_connections 2024-05-08T23:37:33.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.196+0000 7f11146aa700 1 -- 172.21.15.33:0/3249265053 wait complete. 2024-05-08T23:37:33.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.198+0000 7f11146aa700 1 Processor -- start 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.199+0000 7f11146aa700 1 -- start start 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f110c0b3bd0 0x7f110c0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1fc2a0 con 0x7f110c152160 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0ca510 con 0x7f110c0b3bd0 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f11146aa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c14d9e0 con 0x7f110c158020 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f1111c45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f1111c45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52428/0 (socket says 172.21.15.33:52428) 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.201+0000 7f1111c45700 1 -- 172.21.15.33:0/2104296601 learned_addr learned my addr 172.21.15.33:0/2104296601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1112446700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f110c0b3bd0 0x7f110c0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1112c47700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1111c45700 1 -- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 msgr2=0x7f110c0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1111c45700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1111c45700 1 -- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f110c0b3bd0 msgr2=0x7f110c0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1111c45700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f110c0b3bd0 0x7f110c0c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.202+0000 7f1111c45700 1 -- 172.21.15.33:0/2104296601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080094a0 con 0x7f110c152160 2024-05-08T23:37:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.203+0000 7f1111c45700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f110c0730f0 tx=0x7f1108004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.208+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080036a0 con 0x7f110c152160 2024-05-08T23:37:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.208+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1108003820 con 0x7f110c152160 2024-05-08T23:37:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.208+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108003bd0 con 0x7f110c152160 2024-05-08T23:37:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.209+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c1fc4b0 con 0x7f110c152160 2024-05-08T23:37:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.209+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c1fca70 con 0x7f110c152160 2024-05-08T23:37:33.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.210+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c075460 con 0x7f110c152160 2024-05-08T23:37:33.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.233+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1108003d30 con 0x7f110c152160 2024-05-08T23:37:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.233+0000 7f1102ffd700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 0x7f10f006d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.233+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f110808ac40 con 0x7f110c152160 2024-05-08T23:37:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.234+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f110808e2a0 con 0x7f110c152160 2024-05-08T23:37:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.234+0000 7f1112446700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 0x7f10f006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.235+0000 7f1112446700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 0x7f10f006d250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1104005c90 tx=0x7f1104009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:33.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.982+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f110c0ca650 con 0x7f110c152160 2024-05-08T23:37:33.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.986+0000 7f1102ffd700 1 -- 172.21.15.33:0/2104296601 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f1108015080 con 0x7f110c152160 2024-05-08T23:37:33.991 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:33.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.996+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 msgr2=0x7f10f006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:33.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.996+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 0x7f10f006d250 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1104005c90 tx=0x7f1104009040 comp rx=0 tx=0).stop 2024-05-08T23:37:33.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.996+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 msgr2=0x7f110c0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.996+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f110c0730f0 tx=0x7f1108004750 comp rx=0 tx=0).stop 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 shutdown_connections 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10f006ad90 0x7f10f006d250 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f110c158020 0x7f110c0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f110c0b3bd0 0x7f110c0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 --2- 172.21.15.33:0/2104296601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110c152160 0x7f110c0ca9a0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f110c0730f0 tx=0x7f1108004750 comp rx=0 tx=0).stop 2024-05-08T23:37:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:33.997+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 >> 172.21.15.33:0/2104296601 conn(0x7f110c072230 msgr2=0x7f110c1578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:34.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.006+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 shutdown_connections 2024-05-08T23:37:34.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.006+0000 7f11146aa700 1 -- 172.21.15.33:0/2104296601 wait complete. 2024-05-08T23:37:34.040 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:37:34.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:37:34.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:37:34.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.494+0000 7f0eb201d700 1 Processor -- start 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.495+0000 7f0eb201d700 1 -- start start 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.496+0000 7f0eb201d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0eac0c70c0 0x7f0eac0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.496+0000 7f0eb201d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.496+0000 7f0eb201d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac1572f0 con 0x7f0eac150510 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.496+0000 7f0eb201d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac157c30 con 0x7f0eac0c70c0 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.496+0000 7f0eb201d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0c8110 con 0x7f0eac1577a0 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:33264/0 (socket says 172.21.15.33:33264) 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 -- 172.21.15.33:0/1358700885 learned_addr learned my addr 172.21.15.33:0/1358700885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:34.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 -- 172.21.15.33:0/1358700885 >> v2:172.21.15.33:3300/0 conn(0x7f0eac0c70c0 msgr2=0x7f0eac0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 --2- 172.21.15.33:0/1358700885 >> v2:172.21.15.33:3300/0 conn(0x7f0eac0c70c0 0x7f0eac0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 -- 172.21.15.33:0/1358700885 >> v1:172.21.15.42:6789/0 conn(0x7f0eac150510 legacy=0x7f0eac06a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.497+0000 7f0eabfff700 1 -- 172.21.15.33:0/1358700885 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0c7e30 con 0x7f0eac1577a0 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.498+0000 7f0eabfff700 1 --2- 172.21.15.33:0/1358700885 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009e50 tx=0x7f0ea400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67377186b520567f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.498+0000 7f0eaa7fc700 1 -- 172.21.15.33:0/1358700885 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003a20 con 0x7f0eac1577a0 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.498+0000 7f0eaa7fc700 1 -- 172.21.15.33:0/1358700885 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea4003b80 con 0x7f0eac1577a0 2024-05-08T23:37:34.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.498+0000 7f0eaa7fc700 1 -- 172.21.15.33:0/1358700885 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400b980 con 0x7f0eac1577a0 2024-05-08T23:37:34.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.499+0000 7f0eb201d700 1 -- 172.21.15.33:0/1358700885 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 msgr2=0x7f0eac156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.499+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1358700885 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009e50 tx=0x7f0ea400b330 comp rx=0 tx=0).stop 2024-05-08T23:37:34.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.538+0000 7f0eb201d700 1 -- 172.21.15.33:0/1358700885 shutdown_connections 2024-05-08T23:37:34.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.538+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1358700885 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac156db0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.538+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1358700885 >> v2:172.21.15.33:3300/0 conn(0x7f0eac0c70c0 0x7f0eac0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.538+0000 7f0eb201d700 1 -- 172.21.15.33:0/1358700885 >> 172.21.15.33:0/1358700885 conn(0x7f0eac0c8360 msgr2=0x7f0eac0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:34.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.539+0000 7f0eb201d700 1 -- 172.21.15.33:0/1358700885 shutdown_connections 2024-05-08T23:37:34.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.539+0000 7f0eb201d700 1 -- 172.21.15.33:0/1358700885 wait complete. 2024-05-08T23:37:34.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.541+0000 7f0eb201d700 1 Processor -- start 2024-05-08T23:37:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.541+0000 7f0eb201d700 1 -- start start 2024-05-08T23:37:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac0c70c0 0x7f0eac1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 0x7f0eac1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac151f20 con 0x7f0eac0c70c0 2024-05-08T23:37:34.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1f4050 con 0x7f0eac150510 2024-05-08T23:37:34.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.542+0000 7f0eb201d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac1f7f90 con 0x7f0eac1577a0 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.545+0000 7f0eaaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 0x7f0eac1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.545+0000 7f0eaaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 0x7f0eac1f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:35384/0 (socket says 172.21.15.33:35384) 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.545+0000 7f0eaaffd700 1 -- 172.21.15.33:0/1477177851 learned_addr learned my addr 172.21.15.33:0/1477177851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.546+0000 7f0eaaffd700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 msgr2=0x7f0eac0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.546+0000 7f0eaaffd700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0eac1577a0 0x7f0eac0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.546+0000 7f0eaaffd700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac0c70c0 msgr2=0x7f0eac1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.546+0000 7f0eaaffd700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac0c70c0 0x7f0eac1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.546+0000 7f0eaaffd700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4009b30 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.547+0000 7f0eaaffd700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 0x7f0eac1f44e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b4a0 tx=0x7f0e9c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.548+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c011830 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.548+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c011990 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.548+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c011d40 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.548+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac150990 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.548+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac1f3ce0 con 0x7f0eac150510 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.550+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 0x7f0eac14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.550+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac074a20 con 0x7f0eac1f7ac0 2024-05-08T23:37:34.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.551+0000 7f0eab7fe700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 0x7f0eac14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:34.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.552+0000 7f0eab7fe700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0eac0c68c0 con 0x7f0eac1f7ac0 2024-05-08T23:37:34.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.552+0000 7f0eab7fe700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 0x7f0eac14a780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c340 tx=0x7f0ea000c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0016070 con 0x7f0eac1f7ac0 2024-05-08T23:37:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0ea00068e0 con 0x7f0eac1f7ac0 2024-05-08T23:37:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 msgr2=0x7f0eac14a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 0x7f0eac14a780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c340 tx=0x7f0ea000c700 comp rx=0 tx=0).stop 2024-05-08T23:37:34.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 msgr2=0x7f0eac14a780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.553+0000 7f0e9bfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac1f7ac0 0x7f0eac14a780 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c340 tx=0x7f0ea000c700 comp rx=0 tx=0).stop 2024-05-08T23:37:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.559+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0e9c01a400 con 0x7f0eac150510 2024-05-08T23:37:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.559+0000 7f0e9bfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e9006aa50 0x7f0e9006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.560+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0e9c0893e0 con 0x7f0eac150510 2024-05-08T23:37:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.574+0000 7f0eab7fe700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e9006aa50 0x7f0e9006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.574+0000 7f0eab7fe700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e9006aa50 0x7f0e9006cf10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0014040 tx=0x7f0ea0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.580+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 0x7f0eac14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.580+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac073690 con 0x7f0eac14b670 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.580+0000 7f0eabfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 0x7f0eac14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.581+0000 7f0eabfff700 1 -- 172.21.15.33:0/1477177851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0eac0c65e0 con 0x7f0eac14b670 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.581+0000 7f0eabfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 0x7f0eac14acc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4015000 tx=0x7f0ea40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40044f0 con 0x7f0eac14b670 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5112 (secure 0 0 0) 0x7f0ea4004650 con 0x7f0eac14b670 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 msgr2=0x7f0eac14acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 0x7f0eac14acc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4015000 tx=0x7f0ea40095c0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 msgr2=0x7f0eac14acc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.582+0000 7f0e9bfff700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0eac14b670 0x7f0eac14acc0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4015000 tx=0x7f0ea40095c0 comp rx=0 tx=0).stop 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:34.582 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:34.585 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:34.586 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:34.587 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.591 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:34.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 7 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.583+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e9006aa50 msgr2=0x7f0e9006cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.583+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0e9006aa50 0x7f0e9006cf10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0014040 tx=0x7f0ea0009030 comp rx=0 tx=0).stop 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.583+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 msgr2=0x7f0eac1f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.583+0000 7f0eb201d700 1 --2- 172.21.15.33:0/1477177851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eac150510 0x7f0eac1f44e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b4a0 tx=0x7f0e9c00b860 comp rx=0 tx=0).stop 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.584+0000 7f0eabfff700 1 -- 172.21.15.33:0/1477177851 reap_dead start 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.584+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 shutdown_connections 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.584+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 >> 172.21.15.33:0/1477177851 conn(0x7f0eac0c8360 msgr2=0x7f0eac0c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.584+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 shutdown_connections 2024-05-08T23:37:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:34.584+0000 7f0eb201d700 1 -- 172.21.15.33:0/1477177851 wait complete. 2024-05-08T23:37:34.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:34.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:37:34.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:37:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.136+0000 7f782cf21700 1 Processor -- start 2024-05-08T23:37:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.136+0000 7f782cf21700 1 -- start start 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.138+0000 7f782cf21700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78281589d0 0x7f78280c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.138+0000 7f782cf21700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782806ba40 con 0x7f78280c2770 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.138+0000 7f782cf21700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7828070930 con 0x7f78280c7d80 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.138+0000 7f782cf21700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782815f180 con 0x7f78281589d0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.139+0000 7f7825d9b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f78280c2770 0x7f782806a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59286/0 (socket says 172.21.15.33:59286) 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.139+0000 7f7825d9b700 1 -- 172.21.15.33:0/102008293 learned_addr learned my addr 172.21.15.33:0/102008293 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.140+0000 7f782659c700 1 --2- 172.21.15.33:0/102008293 >> v2:172.21.15.33:3300/0 conn(0x7f78281589d0 0x7f78280c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.140+0000 7f782659c700 1 -- 172.21.15.33:0/102008293 >> v1:172.21.15.42:6789/0 conn(0x7f78280c7d80 legacy=0x7f7828070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.140+0000 7f782659c700 1 -- 172.21.15.33:0/102008293 >> v1:172.21.15.33:6789/0 conn(0x7f78280c2770 legacy=0x7f782806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.140+0000 7f782659c700 1 -- 172.21.15.33:0/102008293 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782806b4c0 con 0x7f78281589d0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.142+0000 7f782659c700 1 --2- 172.21.15.33:0/102008293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78281589d0 0x7f78280c21a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7820009d90 tx=0x7f782000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ad2bbd21aea1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.142+0000 7f782559a700 1 -- 172.21.15.33:0/102008293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000e070 con 0x7f78281589d0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.142+0000 7f782559a700 1 -- 172.21.15.33:0/102008293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782000bad0 con 0x7f78281589d0 2024-05-08T23:37:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.142+0000 7f782559a700 1 -- 172.21.15.33:0/102008293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000be60 con 0x7f78281589d0 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.143+0000 7f782cf21700 1 -- 172.21.15.33:0/102008293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78281589d0 msgr2=0x7f78280c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.143+0000 7f782cf21700 1 --2- 172.21.15.33:0/102008293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78281589d0 0x7f78280c21a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7820009d90 tx=0x7f782000b330 comp rx=0 tx=0).stop 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.144+0000 7f782cf21700 1 -- 172.21.15.33:0/102008293 shutdown_connections 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.144+0000 7f782cf21700 1 --2- 172.21.15.33:0/102008293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78281589d0 0x7f78280c21a0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7820009d90 tx=0x7f782000b330 comp rx=0 tx=0).stop 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.144+0000 7f782cf21700 1 -- 172.21.15.33:0/102008293 >> 172.21.15.33:0/102008293 conn(0x7f78280c8360 msgr2=0x7f78280c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.147+0000 7f782cf21700 1 -- 172.21.15.33:0/102008293 shutdown_connections 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.149+0000 7f782cf21700 1 -- 172.21.15.33:0/102008293 wait complete. 2024-05-08T23:37:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.151+0000 7f782cf21700 1 Processor -- start 2024-05-08T23:37:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.151+0000 7f782cf21700 1 -- start start 2024-05-08T23:37:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 0x7f78280c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78280c7d80 0x7f78280c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f78281425b0 0x7f7828142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7828142fa0 con 0x7f78280c2770 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782812e420 con 0x7f78280c7d80 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.152+0000 7f782cf21700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782812e700 con 0x7f78281425b0 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 0x7f78280c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 0x7f78280c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52488/0 (socket says 172.21.15.33:52488) 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 -- 172.21.15.33:0/1213754376 learned_addr learned my addr 172.21.15.33:0/1213754376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f78281425b0 msgr2=0x7f7828142a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f78281425b0 0x7f7828142a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78280c7d80 msgr2=0x7f78280c7430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78280c7d80 0x7f78280c7430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820009a70 con 0x7f78280c2770 2024-05-08T23:37:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.153+0000 7f782659c700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 0x7f78280c6ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7820000f80 tx=0x7f7820004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.154+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820003e30 con 0x7f78280c2770 2024-05-08T23:37:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.154+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782812e9e0 con 0x7f78280c2770 2024-05-08T23:37:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.154+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782812efa0 con 0x7f78280c2770 2024-05-08T23:37:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.155+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820027450 con 0x7f78280c2770 2024-05-08T23:37:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.155+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200277c0 con 0x7f78280c2770 2024-05-08T23:37:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.156+0000 7f782cf21700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 0x7f7808004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.156+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808007f20 con 0x7f7808002a40 2024-05-08T23:37:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.160+0000 7f7825d9b700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 0x7f7808004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.162+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7820019030 con 0x7f78280c2770 2024-05-08T23:37:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.163+0000 7f7816ffd700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f781006ae40 0x7f781006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.163+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f782008b3c0 con 0x7f78280c2770 2024-05-08T23:37:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.168+0000 7f7826d9d700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f781006ae40 0x7f781006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.169+0000 7f7826d9d700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f781006ae40 0x7f781006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f781c007fd0 tx=0x7f781c007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.176+0000 7f7825d9b700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78080080e0 con 0x7f7808002a40 2024-05-08T23:37:35.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.180+0000 7f7825d9b700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 0x7f7808004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f782812dd90 tx=0x7f781800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.186+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818004c80 con 0x7f7808002a40 2024-05-08T23:37:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.189+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7818004de0 con 0x7f7808002a40 2024-05-08T23:37:35.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.189+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 msgr2=0x7f7808004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.189+0000 7f7816ffd700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 0x7f7808004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f782812dd90 tx=0x7f781800ca10 comp rx=0 tx=0).stop 2024-05-08T23:37:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.189+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 msgr2=0x7f7808004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.189+0000 7f7816ffd700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808002a40 0x7f7808004f00 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f782812dd90 tx=0x7f781800ca10 comp rx=0 tx=0).stop 2024-05-08T23:37:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.210+0000 7f782cf21700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 0x7f7808007420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.210+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780800a3e0 con 0x7f7808006fb0 2024-05-08T23:37:35.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.212+0000 7f7825d9b700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 0x7f7808007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.219+0000 7f7825d9b700 1 -- 172.21.15.33:0/1213754376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f780800a5a0 con 0x7f7808006fb0 2024-05-08T23:37:35.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.220+0000 7f7825d9b700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 0x7f7808007420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78280c79f0 tx=0x7f7818000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818020070 con 0x7f7808006fb0 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4457 (secure 0 0 0) 0x7f78180192c0 con 0x7f7808006fb0 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 msgr2=0x7f7808007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 0x7f7808007420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78280c79f0 tx=0x7f7818000ed0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 msgr2=0x7f7808007420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.222+0000 7f7816ffd700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7808006fb0 0x7f7808007420 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78280c79f0 tx=0x7f7818000ed0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 3, 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.227 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.228 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:35.229 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.230 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.235 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.236 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:35.237 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f781006ae40 msgr2=0x7f781006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f781006ae40 0x7f781006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f781c007fd0 tx=0x7f781c007a20 comp rx=0 tx=0).stop 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 msgr2=0x7f78280c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 --2- 172.21.15.33:0/1213754376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78280c2770 0x7f78280c6ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7820000f80 tx=0x7f7820004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f7826d9d700 1 -- 172.21.15.33:0/1213754376 reap_dead start 2024-05-08T23:37:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f7826d9d700 1 -- 172.21.15.33:0/1213754376 reap_dead start 2024-05-08T23:37:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 shutdown_connections 2024-05-08T23:37:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.243+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 >> 172.21.15.33:0/1213754376 conn(0x7f78280c8360 msgr2=0x7f78280ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.244+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 shutdown_connections 2024-05-08T23:37:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.244+0000 7f782cf21700 1 -- 172.21.15.33:0/1213754376 wait complete. 2024-05-08T23:37:35.303 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:35.303 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:37:35.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:37:35.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.773+0000 7fdefffff700 1 Processor -- start 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.773+0000 7fdefffff700 1 -- start start 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.774+0000 7fdefffff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.774+0000 7fdefffff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf00150a60 con 0x7fdf00071170 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.774+0000 7fdefffff700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0006ba60 con 0x7fdf00150680 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.774+0000 7fdefffff700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0015a590 con 0x7fdf000c5e30 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.775+0000 7fdefe7fc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.775+0000 7fdefe7fc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:46802/0 (socket says 172.21.15.33:46802) 2024-05-08T23:37:35.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.775+0000 7fdefe7fc700 1 -- 172.21.15.33:0/1087411320 learned_addr learned my addr 172.21.15.33:0/1087411320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:35.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.776+0000 7fdefe7fc700 1 -- 172.21.15.33:0/1087411320 >> v1:172.21.15.42:6789/0 conn(0x7fdf00150680 legacy=0x7fdf0006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:35.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.776+0000 7fdefe7fc700 1 -- 172.21.15.33:0/1087411320 >> v1:172.21.15.33:6789/0 conn(0x7fdf00071170 legacy=0x7fdf000715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:35.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.776+0000 7fdefe7fc700 1 -- 172.21.15.33:0/1087411320 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf00159910 con 0x7fdf000c5e30 2024-05-08T23:37:35.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.776+0000 7fdefe7fc700 1 --2- 172.21.15.33:0/1087411320 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdef0008920 tx=0x7fdef000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412342a65f4d7f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.780+0000 7fdefdffb700 1 -- 172.21.15.33:0/1087411320 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0010070 con 0x7fdf000c5e30 2024-05-08T23:37:35.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.780+0000 7fdefdffb700 1 -- 172.21.15.33:0/1087411320 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000da40 con 0x7fdf000c5e30 2024-05-08T23:37:35.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.780+0000 7fdefdffb700 1 -- 172.21.15.33:0/1087411320 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000ddd0 con 0x7fdf000c5e30 2024-05-08T23:37:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.790+0000 7fdefffff700 1 -- 172.21.15.33:0/1087411320 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 msgr2=0x7fdf000c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.790+0000 7fdefffff700 1 --2- 172.21.15.33:0/1087411320 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdef0008920 tx=0x7fdef000d330 comp rx=0 tx=0).stop 2024-05-08T23:37:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.806+0000 7fdefffff700 1 -- 172.21.15.33:0/1087411320 shutdown_connections 2024-05-08T23:37:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.806+0000 7fdefffff700 1 --2- 172.21.15.33:0/1087411320 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf000c5860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.806+0000 7fdefffff700 1 -- 172.21.15.33:0/1087411320 >> 172.21.15.33:0/1087411320 conn(0x7fdf000cbbb0 msgr2=0x7fdf000cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.818+0000 7fdefffff700 1 -- 172.21.15.33:0/1087411320 shutdown_connections 2024-05-08T23:37:35.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.818+0000 7fdefffff700 1 -- 172.21.15.33:0/1087411320 wait complete. 2024-05-08T23:37:35.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.821+0000 7fdefffff700 1 Processor -- start 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.825+0000 7fdefffff700 1 -- start start 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 0x7fdf0013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf00144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf00150680 0x7fdf00144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00141960 con 0x7fdf00071170 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0013cbc0 con 0x7fdf00150680 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.826+0000 7fdefffff700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0014aa50 con 0x7fdf000c5e30 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 0x7fdf0013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 0x7fdf0013c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36704/0 (socket says 172.21.15.33:36704) 2024-05-08T23:37:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 -- 172.21.15.33:0/2064045818 learned_addr learned my addr 172.21.15.33:0/2064045818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 msgr2=0x7fdf00144450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdf000c5e30 0x7fdf00144450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf00150680 msgr2=0x7fdf00144990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf00150680 0x7fdf00144990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.827+0000 7fdefeffd700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00085d0 con 0x7fdf00071170 2024-05-08T23:37:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.832+0000 7fdefeffd700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 0x7fdf0013c680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdef800ba50 tx=0x7fdef800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.872+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800cae0 con 0x7fdf00071170 2024-05-08T23:37:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.872+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef800cc40 con 0x7fdf00071170 2024-05-08T23:37:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.872+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8014440 con 0x7fdf00071170 2024-05-08T23:37:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.872+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0014be40 con 0x7fdf00071170 2024-05-08T23:37:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.872+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0014c250 con 0x7fdf00071170 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.883+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdef8006d50 con 0x7fdf00071170 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.884+0000 7fdee77fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee806af20 0x7fdee806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.884+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fdef8088be0 con 0x7fdf00071170 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.884+0000 7fdefe7fc700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee806af20 0x7fdee806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.885+0000 7fdefffff700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 0x7fdee0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.885+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0007f20 con 0x7fdee0002a40 2024-05-08T23:37:35.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.885+0000 7fdefe7fc700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee806af20 0x7fdee806d3e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdef0008920 tx=0x7fdef0012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.893+0000 7fdeff7fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 0x7fdee0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.894+0000 7fdeff7fe700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdee00080e0 con 0x7fdee0002a40 2024-05-08T23:37:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.896+0000 7fdeff7fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 0x7fdee0004f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf0014e380 tx=0x7fdef400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40054e0 con 0x7fdee0002a40 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdef400ce10 con 0x7fdee0002a40 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 msgr2=0x7fdee0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 0x7fdee0004f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf0014e380 tx=0x7fdef400c6a0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 msgr2=0x7fdee0004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.905+0000 7fdee77fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0002a40 0x7fdee0004f00 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf0014e380 tx=0x7fdef400c6a0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.922+0000 7fdefffff700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 0x7fdee0007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.922+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee000a3e0 con 0x7fdee0006fc0 2024-05-08T23:37:35.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.925+0000 7fdeff7fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 0x7fdee0007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.929+0000 7fdeff7fe700 1 -- 172.21.15.33:0/2064045818 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdee000a5a0 con 0x7fdee0006fc0 2024-05-08T23:37:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.934+0000 7fdeff7fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 0x7fdee0007450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdef4013000 tx=0x7fdef4000fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.934+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef401f070 con 0x7fdee0006fc0 2024-05-08T23:37:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.935+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4797 (secure 0 0 0) 0x7fdef4006af0 con 0x7fdee0006fc0 2024-05-08T23:37:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.935+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 msgr2=0x7fdee0007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.935+0000 7fdee77fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 0x7fdee0007450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdef4013000 tx=0x7fdef4000fb0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.935+0000 7fdee77fe700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 msgr2=0x7fdee0007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.935+0000 7fdee77fe700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee0006fc0 0x7fdee0007450 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdef4013000 tx=0x7fdef4000fb0 comp rx=0 tx=0).stop 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 4, 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:35.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.938+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee806af20 msgr2=0x7fdee806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.938+0000 7fdefffff700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdee806af20 0x7fdee806d3e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdef0008920 tx=0x7fdef0012000 comp rx=0 tx=0).stop 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.938+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 msgr2=0x7fdf0013c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.938+0000 7fdefffff700 1 --2- 172.21.15.33:0/2064045818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf00071170 0x7fdf0013c680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdef800ba50 tx=0x7fdef800be10 comp rx=0 tx=0).stop 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.938+0000 7fdeff7fe700 1 -- 172.21.15.33:0/2064045818 reap_dead start 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.941+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 shutdown_connections 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.941+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 >> 172.21.15.33:0/2064045818 conn(0x7fdf000cbbb0 msgr2=0x7fdf0006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.941+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 shutdown_connections 2024-05-08T23:37:35.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:35.941+0000 7fdefffff700 1 -- 172.21.15.33:0/2064045818 wait complete. 2024-05-08T23:37:35.962 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:35.962 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:37:35.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:37:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.254+0000 7feb7eb5c700 1 Processor -- start 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.254+0000 7feb7eb5c700 1 -- start start 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7eb5c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb780c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7eb5c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb78157c30 con 0x7feb78150510 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7eb5c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb780c8110 con 0x7feb781577a0 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7eb5c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb780c7e30 con 0x7feb780c70c0 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7c8f8700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7feb78150510 0x7feb7806a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54330/0 (socket says 172.21.15.33:54330) 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.256+0000 7feb7c8f8700 1 -- 172.21.15.33:0/453798602 learned_addr learned my addr 172.21.15.33:0/453798602 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.257+0000 7feb77fff700 1 --2- 172.21.15.33:0/453798602 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb780c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.258+0000 7feb777fe700 1 -- 172.21.15.33:0/453798602 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259807668 0 0) 0x7feb78157c30 con 0x7feb78150510 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.258+0000 7feb777fe700 1 -- 172.21.15.33:0/453798602 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb600034d0 con 0x7feb78150510 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.259+0000 7feb77fff700 1 -- 172.21.15.33:0/453798602 >> v1:172.21.15.42:6789/0 conn(0x7feb781577a0 legacy=0x7feb78160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.259+0000 7feb77fff700 1 -- 172.21.15.33:0/453798602 >> v1:172.21.15.33:6789/0 conn(0x7feb78150510 legacy=0x7feb7806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.259+0000 7feb77fff700 1 -- 172.21.15.33:0/453798602 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb780c6300 con 0x7feb780c70c0 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.264+0000 7feb77fff700 1 --2- 172.21.15.33:0/453798602 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb780c7780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb70009dc0 tx=0x7feb7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c176832d91658a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.264+0000 7feb777fe700 1 -- 172.21.15.33:0/453798602 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000e070 con 0x7feb780c70c0 2024-05-08T23:37:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.264+0000 7feb777fe700 1 -- 172.21.15.33:0/453798602 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70003a20 con 0x7feb780c70c0 2024-05-08T23:37:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.264+0000 7feb777fe700 1 -- 172.21.15.33:0/453798602 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004d60 con 0x7feb780c70c0 2024-05-08T23:37:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.265+0000 7feb7eb5c700 1 -- 172.21.15.33:0/453798602 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 msgr2=0x7feb780c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.265+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/453798602 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb780c7780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb70009dc0 tx=0x7feb7000b330 comp rx=0 tx=0).stop 2024-05-08T23:37:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.266+0000 7feb7eb5c700 1 -- 172.21.15.33:0/453798602 shutdown_connections 2024-05-08T23:37:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.266+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/453798602 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb780c7780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.266+0000 7feb7eb5c700 1 -- 172.21.15.33:0/453798602 >> 172.21.15.33:0/453798602 conn(0x7feb780c8360 msgr2=0x7feb780c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.266+0000 7feb7eb5c700 1 -- 172.21.15.33:0/453798602 shutdown_connections 2024-05-08T23:37:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.266+0000 7feb7eb5c700 1 -- 172.21.15.33:0/453798602 wait complete. 2024-05-08T23:37:36.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.267+0000 7feb7eb5c700 1 Processor -- start 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.267+0000 7feb7eb5c700 1 -- start start 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.268+0000 7feb7eb5c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb78143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.268+0000 7feb7eb5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb78150510 0x7feb78071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7eb5c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 0x7feb78072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7eb5c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb78142170 con 0x7feb781577a0 2024-05-08T23:37:36.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7eb5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb781451f0 con 0x7feb78150510 2024-05-08T23:37:36.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7eb5c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb78142380 con 0x7feb780c70c0 2024-05-08T23:37:36.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7c8f8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb78143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:36.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7c8f8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb78143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:46840/0 (socket says 172.21.15.33:46840) 2024-05-08T23:37:36.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7c8f8700 1 -- 172.21.15.33:0/161958394 learned_addr learned my addr 172.21.15.33:0/161958394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:36.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.269+0000 7feb7d0f9700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 0x7feb78072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 -- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 msgr2=0x7feb78143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb78143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 -- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb78150510 msgr2=0x7feb78071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb78150510 0x7feb78071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 -- 172.21.15.33:0/161958394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009a70 con 0x7feb781577a0 2024-05-08T23:37:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.270+0000 7feb7d0f9700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 0x7feb78072090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c490 tx=0x7feb6c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.271+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0054e0 con 0x7feb781577a0 2024-05-08T23:37:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.271+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c003680 con 0x7feb781577a0 2024-05-08T23:37:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.271+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c003a30 con 0x7feb781577a0 2024-05-08T23:37:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.272+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb78142660 con 0x7feb781577a0 2024-05-08T23:37:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.272+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb78144e80 con 0x7feb781577a0 2024-05-08T23:37:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.275+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7feb6c003b90 con 0x7feb781577a0 2024-05-08T23:37:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.276+0000 7feb757fa700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 0x7feb6406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.276+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7feb6c088860 con 0x7feb781577a0 2024-05-08T23:37:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.276+0000 7feb7c8f8700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 0x7feb6406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.277+0000 7feb7c8f8700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 0x7feb6406d220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb68009ca0 tx=0x7feb68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.278+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb78071870 con 0x7feb781577a0 2024-05-08T23:37:36.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:36.288+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7feb78071870 con 0x7feb781577a0 2024-05-08T23:37:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.108+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7feb78071870 con 0x7feb781577a0 2024-05-08T23:37:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.109+0000 7feb757fa700 1 -- 172.21.15.33:0/161958394 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7feb78071870 con 0x7feb781577a0 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 msgr2=0x7feb6406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 0x7feb6406d220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb68009ca0 tx=0x7feb68008040 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 msgr2=0x7feb78072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 0x7feb78072090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00c490 tx=0x7feb6c00c7a0 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 shutdown_connections 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7feb6406ad60 0x7feb6406d220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb781577a0 0x7feb78072090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7feb780c70c0 0x7feb78143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 --2- 172.21.15.33:0/161958394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb78150510 0x7feb78071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 >> 172.21.15.33:0/161958394 conn(0x7feb780c8360 msgr2=0x7feb78158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 shutdown_connections 2024-05-08T23:37:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:37.126+0000 7feb7eb5c700 1 -- 172.21.15.33:0/161958394 wait complete. 2024-05-08T23:37:37.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:37:47.174 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:37:47.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:47.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.813+0000 7ff9bb5a5700 1 Processor -- start 2024-05-08T23:37:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.814+0000 7ff9bb5a5700 1 -- start start 2024-05-08T23:37:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.817+0000 7ff9bb5a5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff9b4150510 0x7ff9b40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.817+0000 7ff9bb5a5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b40c70c0 0x7ff9b41577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.817+0000 7ff9bb5a5700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40c8110 con 0x7ff9b4156db0 2024-05-08T23:37:47.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.817+0000 7ff9bb5a5700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40c7e30 con 0x7ff9b4150510 2024-05-08T23:37:47.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.817+0000 7ff9bb5a5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40c6300 con 0x7ff9b40c70c0 2024-05-08T23:37:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.821+0000 7ff9b9b42700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff9b4156db0 0x7ff9b406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:39056/0 (socket says 172.21.15.33:39056) 2024-05-08T23:37:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.821+0000 7ff9b9b42700 1 -- 172.21.15.33:0/1078382020 learned_addr learned my addr 172.21.15.33:0/1078382020 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:47.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.822+0000 7ff9b8b40700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b40c70c0 0x7ff9b41577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:47.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.824+0000 7ff9b9341700 1 --2- 172.21.15.33:0/1078382020 >> v2:172.21.15.42:3300/0 conn(0x7ff9b4150510 0x7ff9b40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.825+0000 7ff9b9341700 1 -- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b40c70c0 msgr2=0x7ff9b41577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.825+0000 7ff9b9341700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b40c70c0 0x7ff9b41577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.825+0000 7ff9b9341700 1 -- 172.21.15.33:0/1078382020 >> v1:172.21.15.42:6789/0 conn(0x7ff9b4156db0 legacy=0x7ff9b406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:47.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.825+0000 7ff9b9341700 1 -- 172.21.15.33:0/1078382020 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40c65e0 con 0x7ff9b4150510 2024-05-08T23:37:47.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.829+0000 7ff9b9341700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4150510 0x7ff9b40c7780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009840 tx=0x7ff9b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c070a1fb437aba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.837+0000 7ff9abfff700 1 -- 172.21.15.33:0/1078382020 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0004230 con 0x7ff9b4150510 2024-05-08T23:37:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.837+0000 7ff9abfff700 1 -- 172.21.15.33:0/1078382020 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0004390 con 0x7ff9b4150510 2024-05-08T23:37:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.837+0000 7ff9abfff700 1 -- 172.21.15.33:0/1078382020 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00036a0 con 0x7ff9b4150510 2024-05-08T23:37:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.838+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4150510 msgr2=0x7ff9b40c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:47.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.838+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4150510 0x7ff9b40c7780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009840 tx=0x7ff9b0009b50 comp rx=0 tx=0).stop 2024-05-08T23:37:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.842+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/1078382020 shutdown_connections 2024-05-08T23:37:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.842+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4150510 0x7ff9b40c7780 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.842+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/1078382020 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b40c70c0 0x7ff9b41577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.842+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/1078382020 >> 172.21.15.33:0/1078382020 conn(0x7ff9b40c8360 msgr2=0x7ff9b40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.843+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/1078382020 shutdown_connections 2024-05-08T23:37:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.843+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/1078382020 wait complete. 2024-05-08T23:37:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.845+0000 7ff9bb5a5700 1 Processor -- start 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.845+0000 7ff9bb5a5700 1 -- start start 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b4150510 0x7ff9b414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4156db0 0x7ff9b40baa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40bb5c0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b41fde70 con 0x7ff9b4156db0 2024-05-08T23:37:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9bb5a5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b41fe110 con 0x7ff9b4150510 2024-05-08T23:37:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9b9341700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9b9341700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33510/0 (socket says 172.21.15.33:33510) 2024-05-08T23:37:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.846+0000 7ff9b9341700 1 -- 172.21.15.33:0/2347105686 learned_addr learned my addr 172.21.15.33:0/2347105686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b8b40700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b4150510 0x7ff9b414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b9341700 1 -- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b4150510 msgr2=0x7ff9b414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:47.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b9341700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b4150510 0x7ff9b414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:47.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b9341700 1 -- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4156db0 msgr2=0x7ff9b40baa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:47.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b9341700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4156db0 0x7ff9b40baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:47.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.847+0000 7ff9b9341700 1 -- 172.21.15.33:0/2347105686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00094a0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.848+0000 7ff9b9341700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009b90 tx=0x7ff9b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.851+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00045c0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.851+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0011630 con 0x7ff9b40c70c0 2024-05-08T23:37:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.851+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00119e0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.852+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b41fe3b0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.853+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b41fe910 con 0x7ff9b40c70c0 2024-05-08T23:37:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.855+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b4075460 con 0x7ff9b40c70c0 2024-05-08T23:37:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.856+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff9b0018050 con 0x7ff9b40c70c0 2024-05-08T23:37:47.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.857+0000 7ff9a9ffb700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 0x7ff99806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:47.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.857+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff9b0089ea0 con 0x7ff9b40c70c0 2024-05-08T23:37:47.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.867+0000 7ff9b8b40700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 0x7ff99806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:47.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.868+0000 7ff9b8b40700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 0x7ff99806d220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4153710 tx=0x7ff9ac009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:47.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:47.872+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff9b0055140 con 0x7ff9b40c70c0 2024-05-08T23:37:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.702+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff9b40b96d0 con 0x7ff9b40c70c0 2024-05-08T23:37:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.703+0000 7ff9a9ffb700 1 -- 172.21.15.33:0/2347105686 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7ff9b002b080 con 0x7ff9b40c70c0 2024-05-08T23:37:48.704 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:48.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.706+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 msgr2=0x7ff99806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:48.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.706+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 0x7ff99806d220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4153710 tx=0x7ff9ac009750 comp rx=0 tx=0).stop 2024-05-08T23:37:48.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.706+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 msgr2=0x7ff9b4153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:48.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.706+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009b90 tx=0x7ff9b0004750 comp rx=0 tx=0).stop 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 shutdown_connections 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff99806ad60 0x7ff99806d220 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9b4156db0 0x7ff9b40baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9b40c70c0 0x7ff9b4153840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 --2- 172.21.15.33:0/2347105686 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff9b4150510 0x7ff9b414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.707+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 >> 172.21.15.33:0/2347105686 conn(0x7ff9b40c8360 msgr2=0x7ff9b4156180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:48.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.708+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 shutdown_connections 2024-05-08T23:37:48.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:48.708+0000 7ff9bb5a5700 1 -- 172.21.15.33:0/2347105686 wait complete. 2024-05-08T23:37:48.735 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:37:48.735 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:37:48.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:37:48.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:37:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.371+0000 7fd5d759e700 1 Processor -- start 2024-05-08T23:37:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.371+0000 7fd5d759e700 1 -- start start 2024-05-08T23:37:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.372+0000 7fd5d759e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.373+0000 7fd5d759e700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd5d815f190 0x7fd5d80c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.373+0000 7fd5d759e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5d806afe0 0x7fd5d8158440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.373+0000 7fd5d759e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d815f5d0 con 0x7fd5d806afe0 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.373+0000 7fd5d759e700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d815e6f0 con 0x7fd5d815f190 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.373+0000 7fd5d759e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80c74d0 con 0x7fd5d806ac00 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.375+0000 7fd5d659c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.375+0000 7fd5d5d9b700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd5d815f190 0x7fd5d80c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.375+0000 7fd5d659c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:49164/0 (socket says 172.21.15.33:49164) 2024-05-08T23:37:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.375+0000 7fd5d659c700 1 -- 172.21.15.33:0/1581000762 learned_addr learned my addr 172.21.15.33:0/1581000762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.376+0000 7fd5d6d9d700 1 --2- 172.21.15.33:0/1581000762 >> v2:172.21.15.33:3300/0 conn(0x7fd5d806afe0 0x7fd5d8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.384+0000 7fd5d659c700 1 -- 172.21.15.33:0/1581000762 >> v2:172.21.15.42:3300/0 conn(0x7fd5d815f190 msgr2=0x7fd5d80c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.384+0000 7fd5d659c700 1 --2- 172.21.15.33:0/1581000762 >> v2:172.21.15.42:3300/0 conn(0x7fd5d815f190 0x7fd5d80c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.384+0000 7fd5d659c700 1 -- 172.21.15.33:0/1581000762 >> v2:172.21.15.33:3300/0 conn(0x7fd5d806afe0 msgr2=0x7fd5d8158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.384+0000 7fd5d659c700 1 --2- 172.21.15.33:0/1581000762 >> v2:172.21.15.33:3300/0 conn(0x7fd5d806afe0 0x7fd5d8158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.384+0000 7fd5d659c700 1 -- 172.21.15.33:0/1581000762 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80c79c0 con 0x7fd5d806ac00 2024-05-08T23:37:49.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.399+0000 7fd5d659c700 1 --2- 172.21.15.33:0/1581000762 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80098f0 tx=0x7fd5c8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585f232d3a81d4e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:49.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.413+0000 7fd5d559a700 1 -- 172.21.15.33:0/1581000762 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800b860 con 0x7fd5d806ac00 2024-05-08T23:37:49.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.413+0000 7fd5d559a700 1 -- 172.21.15.33:0/1581000762 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c800b9c0 con 0x7fd5d806ac00 2024-05-08T23:37:49.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.413+0000 7fd5d559a700 1 -- 172.21.15.33:0/1581000762 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800bd30 con 0x7fd5d806ac00 2024-05-08T23:37:49.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.413+0000 7fd5d759e700 1 -- 172.21.15.33:0/1581000762 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 msgr2=0x7fd5d80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.413+0000 7fd5d759e700 1 --2- 172.21.15.33:0/1581000762 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80098f0 tx=0x7fd5c8009c00 comp rx=0 tx=0).stop 2024-05-08T23:37:49.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 -- 172.21.15.33:0/1581000762 shutdown_connections 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 --2- 172.21.15.33:0/1581000762 >> v2:172.21.15.33:3300/0 conn(0x7fd5d806afe0 0x7fd5d8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 --2- 172.21.15.33:0/1581000762 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806ac00 0x7fd5d80707e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 --2- 172.21.15.33:0/1581000762 >> v2:172.21.15.42:3300/0 conn(0x7fd5d815f190 0x7fd5d80c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 -- 172.21.15.33:0/1581000762 >> 172.21.15.33:0/1581000762 conn(0x7fd5d80cbbb0 msgr2=0x7fd5d80cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 -- 172.21.15.33:0/1581000762 shutdown_connections 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.416+0000 7fd5d759e700 1 -- 172.21.15.33:0/1581000762 wait complete. 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.418+0000 7fd5d759e700 1 Processor -- start 2024-05-08T23:37:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.418+0000 7fd5d759e700 1 -- start start 2024-05-08T23:37:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 0x7fd5d8137c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806afe0 0x7fd5d80c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5d815f190 0x7fd5d815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8152a90 con 0x7fd5d806ac00 2024-05-08T23:37:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d8161f40 con 0x7fd5d815f190 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.420+0000 7fd5d759e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d81417f0 con 0x7fd5d806afe0 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 0x7fd5d8137c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 0x7fd5d8137c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33540/0 (socket says 172.21.15.33:33540) 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 -- 172.21.15.33:0/3019357119 learned_addr learned my addr 172.21.15.33:0/3019357119 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806afe0 msgr2=0x7fd5d80c4310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd5d806afe0 0x7fd5d80c4310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5d815f190 msgr2=0x7fd5d815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5d815f190 0x7fd5d815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.421+0000 7fd5d659c700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8009550 con 0x7fd5d806ac00 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5d659c700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 0x7fd5d8137c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8000c00 tx=0x7fd5c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8004510 con 0x7fd5d806ac00 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d8141ad0 con 0x7fd5d806ac00 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c80036a0 con 0x7fd5d806ac00 2024-05-08T23:37:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8003a50 con 0x7fd5d806ac00 2024-05-08T23:37:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.422+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d8161740 con 0x7fd5d806ac00 2024-05-08T23:37:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.424+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd5c8003bb0 con 0x7fd5d806ac00 2024-05-08T23:37:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.425+0000 7fd5c6ffd700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd5b806ac90 0x7fd5b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.425+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd5c808a4c0 con 0x7fd5d806ac00 2024-05-08T23:37:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.427+0000 7fd5d5d9b700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd5b806ac90 0x7fd5b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.428+0000 7fd5d5d9b700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd5b806ac90 0x7fd5b806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b850 tx=0x7fd5cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.433+0000 7fd5d759e700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 0x7fd5bc004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.433+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc007f20 con 0x7fd5bc002a40 2024-05-08T23:37:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.451+0000 7fd5d6d9d700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 0x7fd5bc004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.452+0000 7fd5d6d9d700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5bc0080e0 con 0x7fd5bc002a40 2024-05-08T23:37:49.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.452+0000 7fd5d6d9d700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 0x7fd5bc004f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0007c70 tx=0x7fd5d000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.456+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000ed60 con 0x7fd5bc002a40 2024-05-08T23:37:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.464+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd5d000eec0 con 0x7fd5bc002a40 2024-05-08T23:37:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.464+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 msgr2=0x7fd5bc004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.464+0000 7fd5c6ffd700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 0x7fd5bc004f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0007c70 tx=0x7fd5d000e3c0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.464+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 msgr2=0x7fd5bc004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.464+0000 7fd5c6ffd700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc002a40 0x7fd5bc004f00 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0007c70 tx=0x7fd5d000e3c0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.495+0000 7fd5d759e700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 0x7fd5bc0072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.495+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc00a480 con 0x7fd5bc006e80 2024-05-08T23:37:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.498+0000 7fd5d6d9d700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 0x7fd5bc0072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.499+0000 7fd5d6d9d700 1 -- 172.21.15.33:0/3019357119 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5bc007830 con 0x7fd5bc006e80 2024-05-08T23:37:49.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.499+0000 7fd5d6d9d700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 0x7fd5bc0072f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0018000 tx=0x7fd5d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0014d90 con 0x7fd5bc006e80 2024-05-08T23:37:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5113 (secure 0 0 0) 0x7fd5d0022070 con 0x7fd5bc006e80 2024-05-08T23:37:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 msgr2=0x7fd5bc0072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 0x7fd5bc0072f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0018000 tx=0x7fd5d0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 msgr2=0x7fd5bc0072f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.503+0000 7fd5c6ffd700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5bc006e80 0x7fd5bc0072f0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0018000 tx=0x7fd5d0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.518 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 17, 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.519 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:49.521 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.522 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:49.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:49.525 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:49.526 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:49.527 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:49.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 7 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:37:49.529 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:49.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:49.531 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:49.531 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd5b806ac90 msgr2=0x7fd5b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd5b806ac90 0x7fd5b806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b850 tx=0x7fd5cc008040 comp rx=0 tx=0).stop 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 msgr2=0x7fd5d8137c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 --2- 172.21.15.33:0/3019357119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d806ac00 0x7fd5d8137c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8000c00 tx=0x7fd5c8004750 comp rx=0 tx=0).stop 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d6d9d700 1 -- 172.21.15.33:0/3019357119 reap_dead start 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 shutdown_connections 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 >> 172.21.15.33:0/3019357119 conn(0x7fd5d80cbbb0 msgr2=0x7fd5d80705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 shutdown_connections 2024-05-08T23:37:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:49.520+0000 7fd5d759e700 1 -- 172.21.15.33:0/3019357119 wait complete. 2024-05-08T23:37:49.601 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:49.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:37:49.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:37:49.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:37:49.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:37:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.061+0000 7fa74fd91700 1 Processor -- start 2024-05-08T23:37:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.064+0000 7fa74fd91700 1 -- start start 2024-05-08T23:37:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 0x7fa7481577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa748156db0 0x7fa74806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480c8110 con 0x7fa7480c70c0 2024-05-08T23:37:50.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7480c7e30 con 0x7fa748156db0 2024-05-08T23:37:50.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.065+0000 7fa74fd91700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480c6300 con 0x7fa748150510 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74db2d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74d32c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 0x7fa7481577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74d32c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 0x7fa7481577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33564/0 (socket says 172.21.15.33:33564) 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74d32c700 1 -- 172.21.15.33:0/1854633039 learned_addr learned my addr 172.21.15.33:0/1854633039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74db2d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:49180/0 (socket says 172.21.15.33:49180) 2024-05-08T23:37:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.069+0000 7fa74e32e700 1 --2- 172.21.15.33:0/1854633039 >> v2:172.21.15.42:3300/0 conn(0x7fa748156db0 0x7fa74806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.070+0000 7fa74e32e700 1 -- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 msgr2=0x7fa7480c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:50.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.070+0000 7fa74e32e700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.070+0000 7fa74e32e700 1 -- 172.21.15.33:0/1854633039 >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 msgr2=0x7fa7481577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:37:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.070+0000 7fa74e32e700 1 --2- 172.21.15.33:0/1854633039 >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 0x7fa7481577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.070+0000 7fa74e32e700 1 -- 172.21.15.33:0/1854633039 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480c65e0 con 0x7fa748156db0 2024-05-08T23:37:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.071+0000 7fa74e32e700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa74806a990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa74806b4b0 tx=0x7fa740009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842fcdfbac3b9055 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.076+0000 7fa74db2d700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:37:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.081+0000 7fa74cb2b700 1 -- 172.21.15.33:0/1854633039 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000ca80 con 0x7fa748156db0 2024-05-08T23:37:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.081+0000 7fa74cb2b700 1 -- 172.21.15.33:0/1854633039 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74000cbe0 con 0x7fa748156db0 2024-05-08T23:37:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.081+0000 7fa74cb2b700 1 -- 172.21.15.33:0/1854633039 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740011400 con 0x7fa748156db0 2024-05-08T23:37:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.082+0000 7fa74fd91700 1 -- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 msgr2=0x7fa74806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.082+0000 7fa74fd91700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa74806a990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa74806b4b0 tx=0x7fa740009f60 comp rx=0 tx=0).stop 2024-05-08T23:37:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.115+0000 7fa74fd91700 1 -- 172.21.15.33:0/1854633039 shutdown_connections 2024-05-08T23:37:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.115+0000 7fa74fd91700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa74806a990 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.115+0000 7fa74fd91700 1 --2- 172.21.15.33:0/1854633039 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa748150510 0x7fa7480c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.115+0000 7fa74fd91700 1 --2- 172.21.15.33:0/1854633039 >> v2:172.21.15.33:3300/0 conn(0x7fa7480c70c0 0x7fa7481577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.115+0000 7fa74fd91700 1 -- 172.21.15.33:0/1854633039 >> 172.21.15.33:0/1854633039 conn(0x7fa7480c8360 msgr2=0x7fa7480c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:50.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.117+0000 7fa74fd91700 1 -- 172.21.15.33:0/1854633039 shutdown_connections 2024-05-08T23:37:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.121+0000 7fa74fd91700 1 -- 172.21.15.33:0/1854633039 wait complete. 2024-05-08T23:37:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.123+0000 7fa74fd91700 1 Processor -- start 2024-05-08T23:37:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.125+0000 7fa74fd91700 1 -- start start 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa7480c70c0 0x7fa7481fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 0x7fa7481e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa7482000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748154c00 con 0x7fa748150510 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7481fcb80 con 0x7fa748156db0 2024-05-08T23:37:50.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.126+0000 7fa74fd91700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7481f2ac0 con 0x7fa7480c70c0 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 0x7fa7481e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 0x7fa7481e1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33578/0 (socket says 172.21.15.33:33578) 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 -- 172.21.15.33:0/3598087564 learned_addr learned my addr 172.21.15.33:0/3598087564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa7480c70c0 msgr2=0x7fa7481fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa7480c70c0 0x7fa7481fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74d32c700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 msgr2=0x7fa7482000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.127+0000 7fa74e32e700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa7482000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.128+0000 7fa74d32c700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748156db0 0x7fa7482000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.128+0000 7fa74d32c700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74000d040 con 0x7fa748150510 2024-05-08T23:37:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.129+0000 7fa74d32c700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 0x7fa7481e1b90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa73800ca50 tx=0x7fa73800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.130+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738011070 con 0x7fa748150510 2024-05-08T23:37:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.131+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73800a960 con 0x7fa748150510 2024-05-08T23:37:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.131+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73800ad10 con 0x7fa748150510 2024-05-08T23:37:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.134+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7481fd100 con 0x7fa748150510 2024-05-08T23:37:50.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.134+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7481fdb30 con 0x7fa748150510 2024-05-08T23:37:50.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.142+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa738015070 con 0x7fa748150510 2024-05-08T23:37:50.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.142+0000 7fa73e7fc700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa72c06ad60 0x7fa72c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.142+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa738089090 con 0x7fa748150510 2024-05-08T23:37:50.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.143+0000 7fa74db2d700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa72c06ad60 0x7fa72c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.144+0000 7fa74db2d700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa72c06ad60 0x7fa72c06d220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa744006fd0 tx=0x7fa744008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.199+0000 7fa74fd91700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 0x7fa748200630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.199+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748074a20 con 0x7fa748155470 2024-05-08T23:37:50.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.199+0000 7fa74e32e700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 0x7fa748200630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.203+0000 7fa74e32e700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa74806d210 con 0x7fa748155470 2024-05-08T23:37:50.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.206+0000 7fa74e32e700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 0x7fa748200630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa74806b4b0 tx=0x7fa740004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74001f070 con 0x7fa748155470 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa740024470 con 0x7fa748155470 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 msgr2=0x7fa748200630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 0x7fa748200630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa74806b4b0 tx=0x7fa740004b40 comp rx=0 tx=0).stop 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 msgr2=0x7fa748200630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.208+0000 7fa73e7fc700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748155470 0x7fa748200630 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa74806b4b0 tx=0x7fa740004b40 comp rx=0 tx=0).stop 2024-05-08T23:37:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.223+0000 7fa74fd91700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 0x7fa748200b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.223+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748157240 con 0x7fa748154fb0 2024-05-08T23:37:50.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.223+0000 7fa74e32e700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 0x7fa748200b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.224+0000 7fa74e32e700 1 -- 172.21.15.33:0/3598087564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7480c6ba0 con 0x7fa748154fb0 2024-05-08T23:37:50.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.224+0000 7fa74e32e700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 0x7fa748200b70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa740009fa0 tx=0x7fa740004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000ca80 con 0x7fa748154fb0 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4628 (secure 0 0 0) 0x7fa74002c8e0 con 0x7fa748154fb0 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 msgr2=0x7fa748200b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 0x7fa748200b70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa740009fa0 tx=0x7fa740004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 msgr2=0x7fa748200b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:50.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.226+0000 7fa73e7fc700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748154fb0 0x7fa748200b70 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa740009fa0 tx=0x7fa740004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.226 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:50.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:50.226 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:50.226 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:37:50.226 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 18, 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:50.227 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:50.228 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:50.229 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:50.232 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:50.233 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:50.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:50.237 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:50.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:50.239 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:50.240 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:50.240 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:50.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:50.240 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:50.240 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.228+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa72c06ad60 msgr2=0x7fa72c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.228+0000 7fa74fd91700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa72c06ad60 0x7fa72c06d220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa744006fd0 tx=0x7fa744008040 comp rx=0 tx=0).stop 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.228+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 msgr2=0x7fa7481e1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.228+0000 7fa74fd91700 1 --2- 172.21.15.33:0/3598087564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa748150510 0x7fa7481e1b90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa73800ca50 tx=0x7fa73800ce10 comp rx=0 tx=0).stop 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.229+0000 7fa74e32e700 1 -- 172.21.15.33:0/3598087564 reap_dead start 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.229+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 shutdown_connections 2024-05-08T23:37:50.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.229+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 >> 172.21.15.33:0/3598087564 conn(0x7fa7480c8360 msgr2=0x7fa74806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:50.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.255+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 shutdown_connections 2024-05-08T23:37:50.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.258+0000 7fa74fd91700 1 -- 172.21.15.33:0/3598087564 wait complete. 2024-05-08T23:37:50.308 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:50.308 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:37:50.308 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:37:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.939+0000 7f240cfe5700 1 Processor -- start 2024-05-08T23:37:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.939+0000 7f240cfe5700 1 -- start start 2024-05-08T23:37:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.941+0000 7f240cfe5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.941+0000 7f240cfe5700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24080c85a0 con 0x7f24080c7330 2024-05-08T23:37:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.941+0000 7f240cfe5700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2408156b40 con 0x7f2408070800 2024-05-08T23:37:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.941+0000 7f240cfe5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2408066dd0 con 0x7f24080c8c50 2024-05-08T23:37:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.943+0000 7f2405d9b700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f2408070800 0x7f2408156fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:39070/0 (socket says 172.21.15.33:39070) 2024-05-08T23:37:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.943+0000 7f2405d9b700 1 -- 172.21.15.33:0/1747182389 learned_addr learned my addr 172.21.15.33:0/1747182389 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.945+0000 7f240559a700 1 -- 172.21.15.33:0/1747182389 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901823952 0 0) 0x7f2408156b40 con 0x7f2408070800 2024-05-08T23:37:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.945+0000 7f240559a700 1 -- 172.21.15.33:0/1747182389 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f00035b0 con 0x7f2408070800 2024-05-08T23:37:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.945+0000 7f240659c700 1 --2- 172.21.15.33:0/1747182389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.946+0000 7f240659c700 1 -- 172.21.15.33:0/1747182389 >> v1:172.21.15.42:6789/0 conn(0x7f2408070800 legacy=0x7f2408156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.946+0000 7f240659c700 1 -- 172.21.15.33:0/1747182389 >> v1:172.21.15.33:6789/0 conn(0x7f24080c7330 legacy=0x7f24080666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.946+0000 7f240659c700 1 -- 172.21.15.33:0/1747182389 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080c87d0 con 0x7f24080c8c50 2024-05-08T23:37:50.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.951+0000 7f240659c700 1 --2- 172.21.15.33:0/1747182389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408071bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2400009dc0 tx=0x7f240000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=badd5a6cc944b760 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.955+0000 7f240559a700 1 -- 172.21.15.33:0/1747182389 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000e070 con 0x7f24080c8c50 2024-05-08T23:37:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.955+0000 7f240559a700 1 -- 172.21.15.33:0/1747182389 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240000bb20 con 0x7f24080c8c50 2024-05-08T23:37:50.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.955+0000 7f240559a700 1 -- 172.21.15.33:0/1747182389 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400012430 con 0x7f24080c8c50 2024-05-08T23:37:50.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.956+0000 7f240cfe5700 1 -- 172.21.15.33:0/1747182389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 msgr2=0x7f2408071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.956+0000 7f240cfe5700 1 --2- 172.21.15.33:0/1747182389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408071bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2400009dc0 tx=0x7f240000b330 comp rx=0 tx=0).stop 2024-05-08T23:37:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.962+0000 7f240cfe5700 1 -- 172.21.15.33:0/1747182389 shutdown_connections 2024-05-08T23:37:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.962+0000 7f240cfe5700 1 --2- 172.21.15.33:0/1747182389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408071bb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.962+0000 7f240cfe5700 1 -- 172.21.15.33:0/1747182389 >> 172.21.15.33:0/1747182389 conn(0x7f240806a990 msgr2=0x7f240806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.962+0000 7f240cfe5700 1 -- 172.21.15.33:0/1747182389 shutdown_connections 2024-05-08T23:37:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.965+0000 7f240cfe5700 1 -- 172.21.15.33:0/1747182389 wait complete. 2024-05-08T23:37:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.967+0000 7f240cfe5700 1 Processor -- start 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.967+0000 7f240cfe5700 1 -- start start 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2408070800 0x7f240814d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24080c7330 0x7f24081f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408159af0 con 0x7f2408070800 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408159f10 con 0x7f24080c7330 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f240cfe5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2408200c70 con 0x7f24080c8c50 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f2406d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f2406d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:49202/0 (socket says 172.21.15.33:49202) 2024-05-08T23:37:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.968+0000 7f2406d9d700 1 -- 172.21.15.33:0/3617641559 learned_addr learned my addr 172.21.15.33:0/3617641559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:50.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.971+0000 7f240659c700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2408070800 0x7f240814d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.972+0000 7f240659c700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 msgr2=0x7f2408158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.972+0000 7f240659c700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24080c8c50 0x7f2408158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.972+0000 7f240659c700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24080c7330 msgr2=0x7f24081f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.972+0000 7f240659c700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24080c7330 0x7f24081f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.972+0000 7f240659c700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2400009a70 con 0x7f2408070800 2024-05-08T23:37:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.973+0000 7f240659c700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2408070800 0x7f240814d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2400016000 tx=0x7f2400004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.973+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400004960 con 0x7f2408070800 2024-05-08T23:37:50.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.973+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400020070 con 0x7f2408070800 2024-05-08T23:37:50.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.973+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000276c0 con 0x7f2408070800 2024-05-08T23:37:50.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.973+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2408157880 con 0x7f2408070800 2024-05-08T23:37:50.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.974+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2408158500 con 0x7f2408070800 2024-05-08T23:37:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.978+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2400027820 con 0x7f2408070800 2024-05-08T23:37:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.979+0000 7f23f6ffd700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f23e806ac90 0x7f23e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.979+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f2400022070 con 0x7f2408070800 2024-05-08T23:37:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.979+0000 7f240cfe5700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 0x7f24081f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.979+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2408074900 con 0x7f24081f3b60 2024-05-08T23:37:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.980+0000 7f2405d9b700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f23e806ac90 0x7f23e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.981+0000 7f2405d9b700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f23e806ac90 0x7f23e806d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f240815a4a0 tx=0x7f23f80079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.983+0000 7f2406d9d700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 0x7f24081f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:50.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.992+0000 7f2406d9d700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24081567d0 con 0x7f24081f3b60 2024-05-08T23:37:50.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.995+0000 7f2406d9d700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 0x7f24081f3f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e700 tx=0x7f23fc00eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:50.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.995+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0052e0 con 0x7f24081f3b60 2024-05-08T23:37:50.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.996+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f23fc005440 con 0x7f24081f3b60 2024-05-08T23:37:50.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.996+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 msgr2=0x7f24081f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:50.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.996+0000 7f23f6ffd700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 0x7f24081f3f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e700 tx=0x7f23fc00eac0 comp rx=0 tx=0).stop 2024-05-08T23:37:50.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.996+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 msgr2=0x7f24081f3f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:50.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:50.996+0000 7f23f6ffd700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081f3b60 0x7f24081f3f40 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e700 tx=0x7f23fc00eac0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.014+0000 7f240cfe5700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 0x7f24081fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.014+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c82e0 con 0x7f24081fcbc0 2024-05-08T23:37:51.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.019+0000 7f2406d9d700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 0x7f24081fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:51.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.025+0000 7f2406d9d700 1 -- 172.21.15.33:0/3617641559 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2408150d40 con 0x7f24081fcbc0 2024-05-08T23:37:51.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.029+0000 7f2406d9d700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 0x7f24081fcfd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0013f0 tx=0x7f23fc001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc021070 con 0x7f24081fcbc0 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4968 (secure 0 0 0) 0x7f23fc007230 con 0x7f24081fcbc0 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 msgr2=0x7f24081fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 0x7f24081fcfd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0013f0 tx=0x7f23fc001120 comp rx=0 tx=0).stop 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 msgr2=0x7f24081fcfd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:37:51.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.038+0000 7f23f6ffd700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f24081fcbc0 0x7f24081fcfd0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0013f0 tx=0x7f23fc001120 comp rx=0 tx=0).stop 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 44, 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 19, 2024-05-08T23:37:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:37:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:37:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:37:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:37:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:37:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:37:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:37:51.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.040+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f23e806ac90 msgr2=0x7f23e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.040+0000 7f240cfe5700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f23e806ac90 0x7f23e806d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f240815a4a0 tx=0x7f23f80079c0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.040+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2408070800 msgr2=0x7f240814d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.040+0000 7f240cfe5700 1 --2- 172.21.15.33:0/3617641559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2408070800 0x7f240814d9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2400016000 tx=0x7f2400004ad0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.041+0000 7f2406d9d700 1 -- 172.21.15.33:0/3617641559 reap_dead start 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.044+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 shutdown_connections 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.044+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 >> 172.21.15.33:0/3617641559 conn(0x7f240806a990 msgr2=0x7f2408154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.048+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 shutdown_connections 2024-05-08T23:37:51.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.049+0000 7f240cfe5700 1 -- 172.21.15.33:0/3617641559 wait complete. 2024-05-08T23:37:51.080 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:37:51.081 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-08T23:37:51.081 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:37:51.081 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-08T23:37:51.081 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:37:51.081 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:37:51.081 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:37:51.112 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:37:51.112 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:37:51.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.727+0000 7f80dcda4700 1 Processor -- start 2024-05-08T23:37:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.731+0000 7f80dcda4700 1 -- start start 2024-05-08T23:37:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.732+0000 7f80dcda4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.733+0000 7f80dcda4700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d80c5c20 con 0x7f80d80d00b0 2024-05-08T23:37:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.733+0000 7f80dcda4700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d8154b50 con 0x7f80d80c4c70 2024-05-08T23:37:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.733+0000 7f80dcda4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80b30b0 con 0x7f80d8154770 2024-05-08T23:37:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.735+0000 7f80d6d9d700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f80d80d00b0 legacy=0x7f80d8164d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:37:51.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.748+0000 7f80d5d9b700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f80d80c4c70 0x7f80d80c60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:39084/0 (socket says 172.21.15.33:39084) 2024-05-08T23:37:51.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.748+0000 7f80d5d9b700 1 -- 172.21.15.33:0/4048398315 learned_addr learned my addr 172.21.15.33:0/4048398315 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:37:51.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.749+0000 7f80d659c700 1 --2- 172.21.15.33:0/4048398315 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:51.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.750+0000 7f80d559a700 1 -- 172.21.15.33:0/4048398315 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641684169 0 0) 0x7f80d8154b50 con 0x7f80d80c4c70 2024-05-08T23:37:51.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.750+0000 7f80d559a700 1 -- 172.21.15.33:0/4048398315 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c00035b0 con 0x7f80d80c4c70 2024-05-08T23:37:51.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.756+0000 7f80d659c700 1 -- 172.21.15.33:0/4048398315 >> v1:172.21.15.42:6789/0 conn(0x7f80d80c4c70 legacy=0x7f80d80c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:51.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.756+0000 7f80d659c700 1 -- 172.21.15.33:0/4048398315 >> v1:172.21.15.33:6789/0 conn(0x7f80d80d00b0 legacy=0x7f80d8164d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:51.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.756+0000 7f80d659c700 1 -- 172.21.15.33:0/4048398315 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80caed0 con 0x7f80d8154770 2024-05-08T23:37:51.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.757+0000 7f80d659c700 1 --2- 172.21.15.33:0/4048398315 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d815ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097f0 tx=0x7f80cc009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0d9df727c7bdb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:51.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.768+0000 7f80d559a700 1 -- 172.21.15.33:0/4048398315 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80016b0 con 0x7f80d8154770 2024-05-08T23:37:51.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.768+0000 7f80d559a700 1 -- 172.21.15.33:0/4048398315 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc004030 con 0x7f80d8154770 2024-05-08T23:37:51.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.768+0000 7f80d559a700 1 -- 172.21.15.33:0/4048398315 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0043c0 con 0x7f80d8154770 2024-05-08T23:37:51.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.769+0000 7f80dcda4700 1 -- 172.21.15.33:0/4048398315 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 msgr2=0x7f80d815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:51.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.769+0000 7f80dcda4700 1 --2- 172.21.15.33:0/4048398315 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d815ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097f0 tx=0x7f80cc009b00 comp rx=0 tx=0).stop 2024-05-08T23:37:51.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.772+0000 7f80dcda4700 1 -- 172.21.15.33:0/4048398315 shutdown_connections 2024-05-08T23:37:51.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.772+0000 7f80dcda4700 1 --2- 172.21.15.33:0/4048398315 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d815ae50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.772+0000 7f80dcda4700 1 -- 172.21.15.33:0/4048398315 >> 172.21.15.33:0/4048398315 conn(0x7f80d8066ae0 msgr2=0x7f80d8066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:51.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.772+0000 7f80dcda4700 1 -- 172.21.15.33:0/4048398315 shutdown_connections 2024-05-08T23:37:51.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.772+0000 7f80dcda4700 1 -- 172.21.15.33:0/4048398315 wait complete. 2024-05-08T23:37:51.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.775+0000 7f80dcda4700 1 Processor -- start 2024-05-08T23:37:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.775+0000 7f80dcda4700 1 -- start start 2024-05-08T23:37:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80d80c4c70 0x7f80d81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 0x7f80d81461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8141fd0 con 0x7f80d80d00b0 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d8158e40 con 0x7f80d80c4c70 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80dcda4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80ce460 con 0x7f80d8154770 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80d5d9b700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 msgr2=0x7f80d81461f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.776+0000 7f80d5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 0x7f80d81461f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:49320/0 (socket says 172.21.15.33:49320) 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 -- 172.21.15.33:0/240820438 learned_addr learned my addr 172.21.15.33:0/240820438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 -- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80d80c4c70 msgr2=0x7f80d81582d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80d80c4c70 0x7f80d81582d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 -- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 msgr2=0x7f80d81461f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 0x7f80d81461f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:51.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.777+0000 7f80d6d9d700 1 -- 172.21.15.33:0/240820438 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0094a0 con 0x7f80d8154770 2024-05-08T23:37:51.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.784+0000 7f80d6d9d700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80d000be50 tx=0x7f80d0009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.788+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cd90 con 0x7f80d8154770 2024-05-08T23:37:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.788+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0016430 con 0x7f80d8154770 2024-05-08T23:37:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.788+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00167c0 con 0x7f80d8154770 2024-05-08T23:37:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.789+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d80cf280 con 0x7f80d8154770 2024-05-08T23:37:51.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.790+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d80cf840 con 0x7f80d8154770 2024-05-08T23:37:51.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.791+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d80732d0 con 0x7f80d8154770 2024-05-08T23:37:51.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.804+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f80d0012020 con 0x7f80d8154770 2024-05-08T23:37:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.804+0000 7f80c6ffd700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 0x7f80b806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:51.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.804+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f80d0089a20 con 0x7f80d8154770 2024-05-08T23:37:51.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.804+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f80d00b69a0 con 0x7f80d8154770 2024-05-08T23:37:51.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.811+0000 7f80d659c700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 0x7f80b806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:51.833+0000 7f80d659c700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 0x7f80b806d100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c00 tx=0x7f80cc008000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:52.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.637+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f80d8075460 con 0x7f80d8154770 2024-05-08T23:37:52.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.645+0000 7f80c6ffd700 1 -- 172.21.15.33:0/240820438 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f80d001b620 con 0x7f80d8154770 2024-05-08T23:37:52.725 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 msgr2=0x7f80b806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 0x7f80b806d100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c00 tx=0x7f80cc008000 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 msgr2=0x7f80d8146730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80d000be50 tx=0x7f80d0009f40 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 shutdown_connections 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f80b806ac40 0x7f80b806d100 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80cc000c00 tx=0x7f80cc008000 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f80d8154770 0x7f80d8146730 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80d80c4c70 0x7f80d81582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 --2- 172.21.15.33:0/240820438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d80d00b0 0x7f80d81461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.735+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 >> 172.21.15.33:0/240820438 conn(0x7f80d8066ae0 msgr2=0x7f80d8152aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.762+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 shutdown_connections 2024-05-08T23:37:52.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:52.762+0000 7f80dcda4700 1 -- 172.21.15.33:0/240820438 wait complete. 2024-05-08T23:37:55.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:37:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.280+0000 7fdef0137700 1 Processor -- start 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.280+0000 7fdef0137700 1 -- start start 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.282+0000 7fdef0137700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdee80c70c0 0x7fdee80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.282+0000 7fdef0137700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 0x7fdee8156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.282+0000 7fdef0137700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee81572f0 con 0x7fdee8150510 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.282+0000 7fdef0137700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee8157c30 con 0x7fdee81577a0 2024-05-08T23:37:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.282+0000 7fdef0137700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80c8110 con 0x7fdee80c70c0 2024-05-08T23:37:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.283+0000 7fdeeded3700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7fdee8150510 legacy=0x7fdee806a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:37:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.283+0000 7fdeee6d4700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 msgr2=0x7fdee8156db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:37:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.284+0000 7fdeee6d4700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 0x7fdee8156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.285+0000 7fdeed6d2700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdee80c70c0 0x7fdee80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.286+0000 7fdeed6d2700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fdee80c70c0 0x7fdee80c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:47108/0 (socket says 172.21.15.33:47108) 2024-05-08T23:37:56.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.286+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1533806240 learned_addr learned my addr 172.21.15.33:0/1533806240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.287+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1533806240 >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 msgr2=0x7fdee8156db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.287+0000 7fdeed6d2700 1 --2- 172.21.15.33:0/1533806240 >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 0x7fdee8156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.287+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1533806240 >> v1:172.21.15.33:6789/0 conn(0x7fdee8150510 legacy=0x7fdee806a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:37:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.287+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1533806240 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80c7e30 con 0x7fdee80c70c0 2024-05-08T23:37:56.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.290+0000 7fdeed6d2700 1 --2- 172.21.15.33:0/1533806240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee80c7780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdee806bbe0 tx=0x7fdee400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9ba7ed0bee8928 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:56.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.298+0000 7fdeeced1700 1 -- 172.21.15.33:0/1533806240 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4010070 con 0x7fdee80c70c0 2024-05-08T23:37:56.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.298+0000 7fdeeced1700 1 -- 172.21.15.33:0/1533806240 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee400da40 con 0x7fdee80c70c0 2024-05-08T23:37:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.300+0000 7fdeeced1700 1 -- 172.21.15.33:0/1533806240 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee400ddd0 con 0x7fdee80c70c0 2024-05-08T23:37:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.301+0000 7fdef0137700 1 -- 172.21.15.33:0/1533806240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 msgr2=0x7fdee80c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:56.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.301+0000 7fdef0137700 1 --2- 172.21.15.33:0/1533806240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee80c7780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdee806bbe0 tx=0x7fdee400d330 comp rx=0 tx=0).stop 2024-05-08T23:37:56.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.303+0000 7fdef0137700 1 -- 172.21.15.33:0/1533806240 shutdown_connections 2024-05-08T23:37:56.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.303+0000 7fdef0137700 1 --2- 172.21.15.33:0/1533806240 >> v2:172.21.15.33:3300/0 conn(0x7fdee81577a0 0x7fdee8156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:56.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.303+0000 7fdef0137700 1 --2- 172.21.15.33:0/1533806240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee80c7780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:56.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.303+0000 7fdef0137700 1 -- 172.21.15.33:0/1533806240 >> 172.21.15.33:0/1533806240 conn(0x7fdee80c8360 msgr2=0x7fdee80c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:56.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.304+0000 7fdef0137700 1 -- 172.21.15.33:0/1533806240 shutdown_connections 2024-05-08T23:37:56.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.304+0000 7fdef0137700 1 -- 172.21.15.33:0/1533806240 wait complete. 2024-05-08T23:37:56.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.306+0000 7fdef0137700 1 Processor -- start 2024-05-08T23:37:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.306+0000 7fdef0137700 1 -- start start 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.307+0000 7fdef0137700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee81f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.307+0000 7fdef0137700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 0x7fdee81f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.307+0000 7fdef0137700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 0x7fdee81f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.307+0000 7fdef0137700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80c47d0 con 0x7fdee81577a0 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdef0137700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8155c00 con 0x7fdee80c70c0 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdef0137700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee81f5ba0 con 0x7fdee8150510 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.308+0000 7fdeee6d4700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 msgr2=0x7fdee81f7f10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdeee6d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 0x7fdee81f7f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.312+0000 7fdeeded3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee81f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdeeded3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee81f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:47114/0 (socket says 172.21.15.33:47114) 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdeeded3700 1 -- 172.21.15.33:0/1869846667 learned_addr learned my addr 172.21.15.33:0/1869846667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:37:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.315+0000 7fdeed6d2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 0x7fdee81f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.317+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 msgr2=0x7fdee81f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.317+0000 7fdeed6d2700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee81f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.317+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 msgr2=0x7fdee81f7f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:37:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.317+0000 7fdeed6d2700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 0x7fdee81f7f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.317+0000 7fdeed6d2700 1 -- 172.21.15.33:0/1869846667 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40085d0 con 0x7fdee8150510 2024-05-08T23:37:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.318+0000 7fdeed6d2700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 0x7fdee81f79d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdee4018000 tx=0x7fdee4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.323+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4009ae0 con 0x7fdee8150510 2024-05-08T23:37:56.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.323+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4009c60 con 0x7fdee8150510 2024-05-08T23:37:56.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.323+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40036a0 con 0x7fdee8150510 2024-05-08T23:37:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.325+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80c4a90 con 0x7fdee8150510 2024-05-08T23:37:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.325+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee81561a0 con 0x7fdee8150510 2024-05-08T23:37:56.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.327+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee8075460 con 0x7fdee8150510 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.333+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdee4003820 con 0x7fdee8150510 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.333+0000 7fdede7fc700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 0x7fdecc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.333+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fdee408a9f0 con 0x7fdee8150510 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.336+0000 7fdeeded3700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 0x7fdecc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.337+0000 7fdeeded3700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 0x7fdecc06d180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdee0007430 tx=0x7fdee0009740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:37:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:56.339+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdee4055cb0 con 0x7fdee8150510 2024-05-08T23:37:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.316+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdee81572f0 con 0x7fdee8150510 2024-05-08T23:37:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.321+0000 7fdede7fc700 1 -- 172.21.15.33:0/1869846667 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fdee81572f0 con 0x7fdee8150510 2024-05-08T23:37:57.320 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.331+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 msgr2=0x7fdecc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.331+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 0x7fdecc06d180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdee0007430 tx=0x7fdee0009740 comp rx=0 tx=0).stop 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.331+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 msgr2=0x7fdee81f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.331+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 0x7fdee81f79d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdee4018000 tx=0x7fdee4004750 comp rx=0 tx=0).stop 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 shutdown_connections 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdecc06acc0 0x7fdecc06d180 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdee0007430 tx=0x7fdee0009740 comp rx=0 tx=0).stop 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdee81577a0 0x7fdee81f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:57.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee80c70c0 0x7fdee81f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:37:57.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 --2- 172.21.15.33:0/1869846667 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdee8150510 0x7fdee81f79d0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdee4018000 tx=0x7fdee4004750 comp rx=0 tx=0).stop 2024-05-08T23:37:57.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.333+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 >> 172.21.15.33:0/1869846667 conn(0x7fdee80c8360 msgr2=0x7fdee806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:37:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.345+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 shutdown_connections 2024-05-08T23:37:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:37:57.345+0000 7fdef0137700 1 -- 172.21.15.33:0/1869846667 wait complete. 2024-05-08T23:38:00.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:38:00.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.757+0000 7f01fbadf700 1 Processor -- start 2024-05-08T23:38:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.759+0000 7f01fbadf700 1 -- start start 2024-05-08T23:38:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.760+0000 7f01fbadf700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.760+0000 7f01fbadf700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f4157c30 con 0x7f01f4150510 2024-05-08T23:38:00.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.760+0000 7f01fbadf700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40c8110 con 0x7f01f41577a0 2024-05-08T23:38:00.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.760+0000 7f01fbadf700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c7e30 con 0x7f01f40c70c0 2024-05-08T23:38:00.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.761+0000 7f01f987b700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f01f4150510 legacy=0x7f01f406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:00.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.767+0000 7f01f907a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.775+0000 7f01f907a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:45702/0 (socket says 172.21.15.33:45702) 2024-05-08T23:38:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.775+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 learned_addr learned my addr 172.21.15.33:0/1819224859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.777+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.777+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.777+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.777+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:00.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.962+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.33:6789/0 conn(0x7f01f4150510 legacy=0x7f01f406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:01.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.978+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:01.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.978+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:01.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.978+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:01.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.978+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:01.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:00.978+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.365+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.33:6789/0 conn(0x7f01f4150510 legacy=0x7f01f406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.387+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:01.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.391+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:01.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.392+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:01.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.392+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:01.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:01.392+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.166+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.33:6789/0 conn(0x7f01f4150510 legacy=0x7f01f406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:02.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.197+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:02.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.197+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:02.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.197+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:02.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.197+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:02.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:02.197+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.760+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01f40c7780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.760+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.760+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.42:6789/0 conn(0x7f01f41577a0 legacy=0x7f01f4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.760+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.33:6789/0 conn(0x7f01f4150510 legacy=0x7f01f406a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.761+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.761+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.761+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40c7e30 con 0x7f01d8006250 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.761+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d8008dc0 con 0x7f01d8003780 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.766+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 msgr2=0x7f01d8008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.766+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.770+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.772+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.772+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.772+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.772+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.968+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 msgr2=0x7f01d8008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.968+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.976+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.977+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.977+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.977+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:03.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:03.977+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:04.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.369+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 msgr2=0x7f01d8008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:04.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.369+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:04.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.386+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.389+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.389+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.389+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:04.389+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.170+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 msgr2=0x7f01d8008710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:05.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.170+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.195+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:05.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.196+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:05.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.196+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:05.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.196+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:05.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:05.196+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:06.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.765+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 msgr2=0x7f01d8005c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.765+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.42:3300/0 conn(0x7f01d8003780 0x7f01d8005c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.765+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 msgr2=0x7f01d8008710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.765+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d8006250 0x7f01d8008710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.765+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01d800b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01eb7fe700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d800eec0 0x7f01d8011350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40c6300 con 0x7f01f41577a0 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40c7e30 con 0x7f01d800eec0 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01eb7fe700 1 -- 172.21.15.33:0/1819224859 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8008dc0 con 0x7f01f40c70c0 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01fa07c700 1 -- 172.21.15.33:0/1819224859 reap_dead start 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01fa07c700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d800eec0 msgr2=0x7f01d8011350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:06.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.766+0000 7f01fa07c700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d800eec0 0x7f01d8011350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.769+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625463612 0 0) 0x7f01f40c6300 con 0x7f01f41577a0 2024-05-08T23:38:06.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.769+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e00035b0 con 0x7f01f41577a0 2024-05-08T23:38:06.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01d800b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441470822 0 0) 0x7f01e00035b0 con 0x7f01f41577a0 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 msgr2=0x7f01d800b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40c70c0 0x7f01d800b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d800eec0 msgr2=0x7f01d8011350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 --2- 172.21.15.33:0/1819224859 >> v2:172.21.15.33:3300/0 conn(0x7f01d800eec0 0x7f01d8011350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4157c30 con 0x7f01f41577a0 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f01ec009900 con 0x7f01f41577a0 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 >> v1:172.21.15.42:6789/0 conn(0x7f01f41577a0 legacy=0x7f01d800d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0003a00 0x7f01e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01e0004850 0x7f01e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0005210 0x7f01e0014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c7e30 con 0x7f01e0003a00 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8008dc0 con 0x7f01e0005210 2024-05-08T23:38:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0014970 con 0x7f01e0004850 2024-05-08T23:38:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f987b700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0003a00 msgr2=0x7f01e0003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.771+0000 7f01f987b700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0003a00 0x7f01e0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01e0004850 0x7f01e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0005210 msgr2=0x7f01e0014220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0005210 0x7f01e0014220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0003a00 msgr2=0x7f01e0003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e0003a00 0x7f01e0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 -- 172.21.15.33:0/1819224859 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4157c30 con 0x7f01e0004850 2024-05-08T23:38:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.772+0000 7f01f907a700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01e0004850 0x7f01e0004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01d8008c70 tx=0x7f01e0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5f7f760fd5b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.781+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec015070 con 0x7f01e0004850 2024-05-08T23:38:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.781+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec019410 con 0x7f01e0004850 2024-05-08T23:38:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.781+0000 7f01f8879700 1 -- 172.21.15.33:0/1819224859 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec009e20 con 0x7f01e0004850 2024-05-08T23:38:06.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.782+0000 7f01fbadf700 1 -- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01e0004850 msgr2=0x7f01e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.782+0000 7f01fbadf700 1 --2- 172.21.15.33:0/1819224859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01e0004850 0x7f01e0004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01d8008c70 tx=0x7f01e0005880 comp rx=0 tx=0).stop 2024-05-08T23:38:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.782+0000 7f01fa07c700 1 -- 172.21.15.33:0/1819224859 reap_dead start 2024-05-08T23:38:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.783+0000 7f01fbadf700 1 -- 172.21.15.33:0/1819224859 shutdown_connections 2024-05-08T23:38:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.783+0000 7f01fbadf700 1 -- 172.21.15.33:0/1819224859 >> 172.21.15.33:0/1819224859 conn(0x7f01f40c8360 msgr2=0x7f01f40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.784+0000 7f01fbadf700 1 -- 172.21.15.33:0/1819224859 shutdown_connections 2024-05-08T23:38:06.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.784+0000 7f01fbadf700 1 -- 172.21.15.33:0/1819224859 wait complete. 2024-05-08T23:38:06.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.786+0000 7f01fbadf700 1 Processor -- start 2024-05-08T23:38:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.787+0000 7f01fbadf700 1 -- start start 2024-05-08T23:38:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0004850 0x7f01f4154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 0x7f01f41ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f41ec5f0 con 0x7f01f4150c80 2024-05-08T23:38:06.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f41eec00 con 0x7f01e0004850 2024-05-08T23:38:06.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fbadf700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c8eb0 con 0x7f01f40cab30 2024-05-08T23:38:06.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01f907a700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 msgr2=0x7f01f41ec0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:06.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01f907a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 0x7f01f41ec0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:06.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fa07c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:06.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fa07c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:59060/0 (socket says 172.21.15.33:59060) 2024-05-08T23:38:06.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.788+0000 7f01fa07c700 1 -- 172.21.15.33:0/994484497 learned_addr learned my addr 172.21.15.33:0/994484497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.793+0000 7f01fa07c700 1 -- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0004850 msgr2=0x7f01f4154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.793+0000 7f01fa07c700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0004850 0x7f01f4154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.793+0000 7f01fa07c700 1 -- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 msgr2=0x7f01f41ec0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.793+0000 7f01fa07c700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 0x7f01f41ec0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.793+0000 7f01fa07c700 1 -- 172.21.15.33:0/994484497 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4151130 con 0x7f01f40cab30 2024-05-08T23:38:06.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.799+0000 7f01fa07c700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01f406fdf0 tx=0x7f01e400c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:06.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.804+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4004220 con 0x7f01f40cab30 2024-05-08T23:38:06.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.804+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4004380 con 0x7f01f40cab30 2024-05-08T23:38:06.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.804+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40072f0 con 0x7f01f40cab30 2024-05-08T23:38:06.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.804+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f41546e0 con 0x7f01f40cab30 2024-05-08T23:38:06.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.804+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41eee50 con 0x7f01f40cab30 2024-05-08T23:38:06.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.806+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f41508f0 con 0x7f01f40cab30 2024-05-08T23:38:06.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.821+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f01e4007450 con 0x7f01f40cab30 2024-05-08T23:38:06.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.822+0000 7f01ea7fc700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 0x7f01d806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:06.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.822+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f01e4088880 con 0x7f01f40cab30 2024-05-08T23:38:06.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.822+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f01e40b59a0 con 0x7f01f40cab30 2024-05-08T23:38:06.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.823+0000 7f01f987b700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 0x7f01d806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:06.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:06.825+0000 7f01f987b700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 0x7f01d806e240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c9390 tx=0x7f01f0009d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:07.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.621+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f01f4075460 con 0x7f01f40cab30 2024-05-08T23:38:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.625+0000 7f01ea7fc700 1 -- 172.21.15.33:0/994484497 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f01e4020080 con 0x7f01f40cab30 2024-05-08T23:38:07.635 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":46,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:38:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 msgr2=0x7f01d806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 0x7f01d806e240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c9390 tx=0x7f01f0009d30 comp rx=0 tx=0).stop 2024-05-08T23:38:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 msgr2=0x7f01f41ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01f406fdf0 tx=0x7f01e400c720 comp rx=0 tx=0).stop 2024-05-08T23:38:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 shutdown_connections 2024-05-08T23:38:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f01d806bd80 0x7f01d806e240 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f01f40c9390 tx=0x7f01f0009d30 comp rx=0 tx=0).stop 2024-05-08T23:38:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f01f40cab30 0x7f01f41ef370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01e0004850 0x7f01f4154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 --2- 172.21.15.33:0/994484497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4150c80 0x7f01f41ec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.742+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 >> 172.21.15.33:0/994484497 conn(0x7f01f40c8360 msgr2=0x7f01f414e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:07.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.755+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 shutdown_connections 2024-05-08T23:38:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:07.761+0000 7f01fbadf700 1 -- 172.21.15.33:0/994484497 wait complete. 2024-05-08T23:38:07.795 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:38:07.795 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:38:07.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:38:08.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.297+0000 7ff49cc34700 1 Processor -- start 2024-05-08T23:38:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.297+0000 7ff49cc34700 1 -- start start 2024-05-08T23:38:08.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.298+0000 7ff49cc34700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff498158c00 0x7ff4980d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.298+0000 7ff49cc34700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff498161db0 con 0x7ff4980c4c60 2024-05-08T23:38:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.298+0000 7ff49cc34700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff498162930 con 0x7ff4980cbd50 2024-05-08T23:38:08.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.298+0000 7ff49cc34700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff498158fe0 con 0x7ff498158c00 2024-05-08T23:38:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.301+0000 7ff496d9d700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7ff4980cbd50 0x7ff49815e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:43978/0 (socket says 172.21.15.33:43978) 2024-05-08T23:38:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.301+0000 7ff496d9d700 1 -- 172.21.15.33:0/4176947437 learned_addr learned my addr 172.21.15.33:0/4176947437 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:08.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.303+0000 7ff49659c700 1 --2- 172.21.15.33:0/4176947437 >> v2:172.21.15.42:3300/0 conn(0x7ff498158c00 0x7ff4980d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.305+0000 7ff49559a700 1 -- 172.21.15.33:0/4176947437 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980541828 0 0) 0x7ff498162930 con 0x7ff4980cbd50 2024-05-08T23:38:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.305+0000 7ff49559a700 1 -- 172.21.15.33:0/4176947437 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4800035b0 con 0x7ff4980cbd50 2024-05-08T23:38:08.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.306+0000 7ff495d9b700 1 -- 172.21.15.33:0/4176947437 >> v1:172.21.15.33:6789/0 conn(0x7ff4980c4c60 legacy=0x7ff4980cb640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.307+0000 7ff49659c700 1 -- 172.21.15.33:0/4176947437 >> v1:172.21.15.42:6789/0 conn(0x7ff4980cbd50 legacy=0x7ff49815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.307+0000 7ff49659c700 1 -- 172.21.15.33:0/4176947437 >> v1:172.21.15.33:6789/0 conn(0x7ff4980c4c60 legacy=0x7ff4980cb640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.307+0000 7ff49659c700 1 -- 172.21.15.33:0/4176947437 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980cc1d0 con 0x7ff498158c00 2024-05-08T23:38:08.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.309+0000 7ff49659c700 1 --2- 172.21.15.33:0/4176947437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff498158c00 0x7ff4980d0060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff490009d90 tx=0x7ff49000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25020839bccc7661 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:08.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.311+0000 7ff49559a700 1 -- 172.21.15.33:0/4176947437 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000e070 con 0x7ff498158c00 2024-05-08T23:38:08.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.311+0000 7ff49559a700 1 -- 172.21.15.33:0/4176947437 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49000bae0 con 0x7ff498158c00 2024-05-08T23:38:08.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.311+0000 7ff49559a700 1 -- 172.21.15.33:0/4176947437 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000be90 con 0x7ff498158c00 2024-05-08T23:38:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.312+0000 7ff49cc34700 1 -- 172.21.15.33:0/4176947437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff498158c00 msgr2=0x7ff4980d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.312+0000 7ff49cc34700 1 --2- 172.21.15.33:0/4176947437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff498158c00 0x7ff4980d0060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff490009d90 tx=0x7ff49000b330 comp rx=0 tx=0).stop 2024-05-08T23:38:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.313+0000 7ff49cc34700 1 -- 172.21.15.33:0/4176947437 shutdown_connections 2024-05-08T23:38:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.313+0000 7ff49cc34700 1 --2- 172.21.15.33:0/4176947437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff498158c00 0x7ff4980d0060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.313+0000 7ff49cc34700 1 -- 172.21.15.33:0/4176947437 >> 172.21.15.33:0/4176947437 conn(0x7ff498066ae0 msgr2=0x7ff498066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:08.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.313+0000 7ff49cc34700 1 -- 172.21.15.33:0/4176947437 shutdown_connections 2024-05-08T23:38:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.313+0000 7ff49cc34700 1 -- 172.21.15.33:0/4176947437 wait complete. 2024-05-08T23:38:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.315+0000 7ff49cc34700 1 Processor -- start 2024-05-08T23:38:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.315+0000 7ff49cc34700 1 -- start start 2024-05-08T23:38:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4980c4c60 0x7ff4980ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 0x7ff49814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff498158c00 0x7ff49815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498161ba0 con 0x7ff498158c00 2024-05-08T23:38:08.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981f4990 con 0x7ff4980c4c60 2024-05-08T23:38:08.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.317+0000 7ff49cc34700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981f4ec0 con 0x7ff4980cbd50 2024-05-08T23:38:08.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.319+0000 7ff496d9d700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff498158c00 msgr2=0x7ff49815d7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:08.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.319+0000 7ff496d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff498158c00 0x7ff49815d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:08.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.319+0000 7ff495d9b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 0x7ff49814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:08.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.319+0000 7ff495d9b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 0x7ff49814d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-05-08T23:38:08.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.319+0000 7ff495d9b700 1 -- 172.21.15.33:0/2637172254 learned_addr learned my addr 172.21.15.33:0/2637172254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:08.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.320+0000 7ff495d9b700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4980c4c60 msgr2=0x7ff4980ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.320+0000 7ff495d9b700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4980c4c60 0x7ff4980ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:08.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.320+0000 7ff495d9b700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff498158c00 msgr2=0x7ff49815d7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:08.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.320+0000 7ff495d9b700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff498158c00 0x7ff49815d7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:08.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.320+0000 7ff495d9b700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490009a70 con 0x7ff4980cbd50 2024-05-08T23:38:08.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.322+0000 7ff495d9b700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 0x7ff49814d3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b580 tx=0x7ff48800b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:08.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.323+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488003e50 con 0x7ff4980cbd50 2024-05-08T23:38:08.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.324+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488005070 con 0x7ff4980cbd50 2024-05-08T23:38:08.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.324+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488005420 con 0x7ff4980cbd50 2024-05-08T23:38:08.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.324+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4981f4be0 con 0x7ff4980cbd50 2024-05-08T23:38:08.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.324+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff498161830 con 0x7ff4980cbd50 2024-05-08T23:38:08.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.328+0000 7ff49cc34700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 0x7ff47c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.328+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c007f20 con 0x7ff47c002a40 2024-05-08T23:38:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.331+0000 7ff49659c700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 0x7ff47c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.342+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff488005580 con 0x7ff4980cbd50 2024-05-08T23:38:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.343+0000 7ff486ffd700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff47806ac40 0x7ff47806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.343+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff488088810 con 0x7ff4980cbd50 2024-05-08T23:38:08.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.344+0000 7ff496d9d700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff47806ac40 0x7ff47806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:08.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.345+0000 7ff496d9d700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff47806ac40 0x7ff47806d100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c007980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:08.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.346+0000 7ff49659c700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff47c0080e0 con 0x7ff47c002a40 2024-05-08T23:38:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.350+0000 7ff49659c700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 0x7ff47c004f00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff490009d90 tx=0x7ff4900040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490004610 con 0x7ff47c002a40 2024-05-08T23:38:08.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff490004770 con 0x7ff47c002a40 2024-05-08T23:38:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 msgr2=0x7ff47c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 0x7ff47c004f00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff490009d90 tx=0x7ff4900040e0 comp rx=0 tx=0).stop 2024-05-08T23:38:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 msgr2=0x7ff47c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:08.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.351+0000 7ff486ffd700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c002a40 0x7ff47c004f00 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff490009d90 tx=0x7ff4900040e0 comp rx=0 tx=0).stop 2024-05-08T23:38:08.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.401+0000 7ff49cc34700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 0x7ff47c007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:08.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.401+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c00a3e0 con 0x7ff47c006f70 2024-05-08T23:38:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.406+0000 7ff49659c700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 0x7ff47c007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.413+0000 7ff49659c700 1 -- 172.21.15.33:0/2637172254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff47c00a5a0 con 0x7ff47c006f70 2024-05-08T23:38:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.414+0000 7ff49659c700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 0x7ff47c007400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0080b0 tx=0x7ff490012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49002d070 con 0x7ff47c006f70 2024-05-08T23:38:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4931 (secure 0 0 0) 0x7ff49001e6c0 con 0x7ff47c006f70 2024-05-08T23:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 msgr2=0x7ff47c007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 0x7ff47c007400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0080b0 tx=0x7ff490012000 comp rx=0 tx=0).stop 2024-05-08T23:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 msgr2=0x7ff47c007400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.418+0000 7ff486ffd700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff47c006f70 0x7ff47c007400 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0080b0 tx=0x7ff490012000 comp rx=0 tx=0).stop 2024-05-08T23:38:08.420 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 46, 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:08.421 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:08.422 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:08.425 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.427 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:08.431 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:08.432 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff47806ac40 msgr2=0x7ff47806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff47806ac40 0x7ff47806d100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c007980 comp rx=0 tx=0).stop 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 msgr2=0x7ff49814d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 --2- 172.21.15.33:0/2637172254 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff4980cbd50 0x7ff49814d3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b580 tx=0x7ff48800b940 comp rx=0 tx=0).stop 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff496d9d700 1 -- 172.21.15.33:0/2637172254 reap_dead start 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff496d9d700 1 -- 172.21.15.33:0/2637172254 reap_dead start 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 shutdown_connections 2024-05-08T23:38:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.428+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 >> 172.21.15.33:0/2637172254 conn(0x7ff498066ae0 msgr2=0x7ff4980c50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:08.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.443+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 shutdown_connections 2024-05-08T23:38:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:08.444+0000 7ff49cc34700 1 -- 172.21.15.33:0/2637172254 wait complete. 2024-05-08T23:38:08.505 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:38:08.505 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:38:08.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:38:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.028+0000 7fcb2b519700 1 Processor -- start 2024-05-08T23:38:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.029+0000 7fcb2b519700 1 -- start start 2024-05-08T23:38:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcb24151470 0x7fcb2406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3330 0x7fcb2406c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 0x7fcb2415a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb24151850 con 0x7fcb240c3880 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240cbc90 con 0x7fcb24151470 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.030+0000 7fcb2b519700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2415b510 con 0x7fcb240c3330 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb29ab6700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 msgr2=0x7fcb2415a640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb29ab6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 0x7fcb2415a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcb24151470 0x7fcb2406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fcb24151470 0x7fcb2406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:48284/0 (socket says 172.21.15.33:48284) 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 -- 172.21.15.33:0/1271371926 learned_addr learned my addr 172.21.15.33:0/1271371926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 -- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3330 msgr2=0x7fcb2406c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 --2- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3330 0x7fcb2406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 -- 172.21.15.33:0/1271371926 >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 msgr2=0x7fcb2415a640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 --2- 172.21.15.33:0/1271371926 >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 0x7fcb2415a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 -- 172.21.15.33:0/1271371926 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240cb190 con 0x7fcb24151470 2024-05-08T23:38:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.031+0000 7fcb292b5700 1 --2- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb24151470 0x7fcb2406bce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb240691b0 tx=0x7fcb20009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4525a4d5403bf218 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.032+0000 7fcb1bfff700 1 -- 172.21.15.33:0/1271371926 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000b870 con 0x7fcb24151470 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.032+0000 7fcb1bfff700 1 -- 172.21.15.33:0/1271371926 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000b9d0 con 0x7fcb24151470 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.032+0000 7fcb1bfff700 1 -- 172.21.15.33:0/1271371926 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000bd60 con 0x7fcb24151470 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 -- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb24151470 msgr2=0x7fcb2406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 --2- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb24151470 0x7fcb2406bce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcb240691b0 tx=0x7fcb20009bd0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 -- 172.21.15.33:0/1271371926 shutdown_connections 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 --2- 172.21.15.33:0/1271371926 >> v2:172.21.15.33:3300/0 conn(0x7fcb240c3880 0x7fcb2415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 --2- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb24151470 0x7fcb2406bce0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 --2- 172.21.15.33:0/1271371926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3330 0x7fcb2406c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.033+0000 7fcb2b519700 1 -- 172.21.15.33:0/1271371926 >> 172.21.15.33:0/1271371926 conn(0x7fcb240c9480 msgr2=0x7fcb240c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.042+0000 7fcb2b519700 1 -- 172.21.15.33:0/1271371926 shutdown_connections 2024-05-08T23:38:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.042+0000 7fcb2b519700 1 -- 172.21.15.33:0/1271371926 wait complete. 2024-05-08T23:38:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.045+0000 7fcb2b519700 1 Processor -- start 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.046+0000 7fcb2b519700 1 -- start start 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb240c3330 0x7fcb241ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 0x7fcb24150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb24151470 0x7fcb241508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241f0890 con 0x7fcb24151470 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241f0310 con 0x7fcb240c3330 2024-05-08T23:38:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb2b519700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241ede60 con 0x7fcb240c3880 2024-05-08T23:38:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb29ab6700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb24151470 msgr2=0x7fcb241508d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.047+0000 7fcb29ab6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb24151470 0x7fcb241508d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.051+0000 7fcb28ab4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 0x7fcb24150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.051+0000 7fcb28ab4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 0x7fcb24150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:59084/0 (socket says 172.21.15.33:59084) 2024-05-08T23:38:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.051+0000 7fcb28ab4700 1 -- 172.21.15.33:0/41542264 learned_addr learned my addr 172.21.15.33:0/41542264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.053+0000 7fcb292b5700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb240c3330 0x7fcb241ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb240c3330 msgr2=0x7fcb241ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb240c3330 0x7fcb241ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb24151470 msgr2=0x7fcb241508d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb24151470 0x7fcb241508d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20009550 con 0x7fcb240c3880 2024-05-08T23:38:09.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.060+0000 7fcb28ab4700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 0x7fcb24150390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b4a0 tx=0x7fcb1400b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:09.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.081+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14011830 con 0x7fcb240c3880 2024-05-08T23:38:09.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.081+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb14011990 con 0x7fcb240c3880 2024-05-08T23:38:09.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.081+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14011d40 con 0x7fcb240c3880 2024-05-08T23:38:09.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.085+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb241ef3d0 con 0x7fcb240c3880 2024-05-08T23:38:09.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.085+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb241ee100 con 0x7fcb240c3880 2024-05-08T23:38:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.091+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb1401a400 con 0x7fcb240c3880 2024-05-08T23:38:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.091+0000 7fcb19ffb700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb0806ace0 0x7fcb0806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.091+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fcb14089180 con 0x7fcb240c3880 2024-05-08T23:38:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.096+0000 7fcb292b5700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb0806ace0 0x7fcb0806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.097+0000 7fcb292b5700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb0806ace0 0x7fcb0806d1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c00 tx=0x7fcb20008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.103+0000 7fcb2b519700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 0x7fcb241fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.103+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb24074a20 con 0x7fcb241eeb60 2024-05-08T23:38:09.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.105+0000 7fcb29ab6700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 0x7fcb241fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.106+0000 7fcb29ab6700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb240cc210 con 0x7fcb241eeb60 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.112+0000 7fcb29ab6700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 0x7fcb241fff00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c015470 con 0x7fcb241eeb60 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcb1c0155d0 con 0x7fcb241eeb60 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 msgr2=0x7fcb241fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 0x7fcb241fff00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c009e90 comp rx=0 tx=0).stop 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 msgr2=0x7fcb241fff00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.113+0000 7fcb19ffb700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb241eeb60 0x7fcb241fff00 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c009e90 comp rx=0 tx=0).stop 2024-05-08T23:38:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.141+0000 7fcb2b519700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 0x7fcb24200440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.141+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240cbf30 con 0x7fcb240c2e10 2024-05-08T23:38:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.152+0000 7fcb29ab6700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 0x7fcb24200440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:09.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.160+0000 7fcb29ab6700 1 -- 172.21.15.33:0/41542264 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb240cb750 con 0x7fcb240c2e10 2024-05-08T23:38:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.164+0000 7fcb29ab6700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 0x7fcb24200440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c01b960 con 0x7fcb240c2e10 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7fcb1c023410 con 0x7fcb240c2e10 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 msgr2=0x7fcb24200440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 0x7fcb24200440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c003cd0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 msgr2=0x7fcb24200440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.168+0000 7fcb19ffb700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c2e10 0x7fcb24200440 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c003d90 tx=0x7fcb1c003cd0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.167 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 46, 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 3, 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:09.180 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.173+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb0806ace0 msgr2=0x7fcb0806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.173+0000 7fcb2b519700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb0806ace0 0x7fcb0806d1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c00 tx=0x7fcb20008030 comp rx=0 tx=0).stop 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.173+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 msgr2=0x7fcb24150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.173+0000 7fcb2b519700 1 --2- 172.21.15.33:0/41542264 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb240c3880 0x7fcb24150390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b4a0 tx=0x7fcb1400b7b0 comp rx=0 tx=0).stop 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.174+0000 7fcb29ab6700 1 -- 172.21.15.33:0/41542264 reap_dead start 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.174+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 shutdown_connections 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.174+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 >> 172.21.15.33:0/41542264 conn(0x7fcb240c9480 msgr2=0x7fcb2406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.177+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 shutdown_connections 2024-05-08T23:38:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:09.177+0000 7fcb2b519700 1 -- 172.21.15.33:0/41542264 wait complete. 2024-05-08T23:38:09.212 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:38:09.212 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:38:20.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:38:20.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:38:29.213 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:38:29.213 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-08T23:38:29.213 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:38:29.214 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:38:29.219 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:38:29.219 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:38:29.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:38:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.703+0000 7ff6593ea700 1 Processor -- start 2024-05-08T23:38:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.703+0000 7ff6593ea700 1 -- start start 2024-05-08T23:38:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.705+0000 7ff6593ea700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff654150510 0x7ff6540c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.705+0000 7ff6593ea700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 0x7ff6541577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.705+0000 7ff6593ea700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6540c8110 con 0x7ff654156db0 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.705+0000 7ff6593ea700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6540c7e30 con 0x7ff6540c70c0 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.705+0000 7ff6593ea700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6540c6300 con 0x7ff654150510 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.707+0000 7ff6537fe700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7ff654156db0 legacy=0x7ff65406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.707+0000 7ff6527fc700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 msgr2=0x7ff6541577a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:38:29.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.707+0000 7ff6527fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 0x7ff6541577a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:29.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.713+0000 7ff652ffd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff654150510 0x7ff6540c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.713+0000 7ff652ffd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff654150510 0x7ff6540c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46700/0 (socket says 172.21.15.33:46700) 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.713+0000 7ff652ffd700 1 -- 172.21.15.33:0/1120882391 learned_addr learned my addr 172.21.15.33:0/1120882391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.721+0000 7ff652ffd700 1 -- 172.21.15.33:0/1120882391 >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 msgr2=0x7ff6541577a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.721+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1120882391 >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 0x7ff6541577a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.721+0000 7ff652ffd700 1 -- 172.21.15.33:0/1120882391 >> v1:172.21.15.33:6789/0 conn(0x7ff654156db0 legacy=0x7ff65406a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.721+0000 7ff652ffd700 1 -- 172.21.15.33:0/1120882391 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6540c65e0 con 0x7ff654150510 2024-05-08T23:38:29.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.722+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1120882391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654150510 0x7ff6540c7780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff65406f310 tx=0x7ff64c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420e9dec930af638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.734+0000 7ff651ffb700 1 -- 172.21.15.33:0/1120882391 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c00e070 con 0x7ff654150510 2024-05-08T23:38:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.734+0000 7ff651ffb700 1 -- 172.21.15.33:0/1120882391 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c003a20 con 0x7ff654150510 2024-05-08T23:38:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.734+0000 7ff651ffb700 1 -- 172.21.15.33:0/1120882391 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c003dd0 con 0x7ff654150510 2024-05-08T23:38:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.738+0000 7ff6593ea700 1 -- 172.21.15.33:0/1120882391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654150510 msgr2=0x7ff6540c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.738+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1120882391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654150510 0x7ff6540c7780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff65406f310 tx=0x7ff64c00b330 comp rx=0 tx=0).stop 2024-05-08T23:38:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.770+0000 7ff6593ea700 1 -- 172.21.15.33:0/1120882391 shutdown_connections 2024-05-08T23:38:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.770+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1120882391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654150510 0x7ff6540c7780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.770+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1120882391 >> v2:172.21.15.33:3300/0 conn(0x7ff6540c70c0 0x7ff6541577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:29.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.770+0000 7ff6593ea700 1 -- 172.21.15.33:0/1120882391 >> 172.21.15.33:0/1120882391 conn(0x7ff6540c8360 msgr2=0x7ff6540c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.812+0000 7ff6593ea700 1 -- 172.21.15.33:0/1120882391 shutdown_connections 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.812+0000 7ff6593ea700 1 -- 172.21.15.33:0/1120882391 wait complete. 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.814+0000 7ff6593ea700 1 Processor -- start 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.815+0000 7ff6593ea700 1 -- start start 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff654150510 0x7ff6541fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 0x7ff654155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541f6bd0 con 0x7ff6540c70c0 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541510a0 con 0x7ff654156db0 2024-05-08T23:38:29.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff6593ea700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff654151380 con 0x7ff654150510 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41928/0 (socket says 172.21.15.33:41928) 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 -- 172.21.15.33:0/1322462741 learned_addr learned my addr 172.21.15.33:0/1322462741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 msgr2=0x7ff654155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 msgr2=0x7ff654155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:29.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.817+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:29.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.820+0000 7ff6527fc700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff654150510 0x7ff6541fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.821+0000 7ff6537fe700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 0x7ff654155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.830+0000 7ff6537fe700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff654150510 msgr2=0x7ff6541fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.830+0000 7ff6537fe700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff654150510 0x7ff6541fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.830+0000 7ff6537fe700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 msgr2=0x7ff654155410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.830+0000 7ff6537fe700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.830+0000 7ff6537fe700 1 -- 172.21.15.33:0/1322462741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c009b30 con 0x7ff654156db0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.838+0000 7ff6537fe700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 0x7ff654155e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff644009fd0 tx=0x7ff64400b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.838+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644003e50 con 0x7ff654156db0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.838+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64400be70 con 0x7ff654156db0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.838+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440062f0 con 0x7ff654156db0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.839+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654151660 con 0x7ff654156db0 2024-05-08T23:38:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.839+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff654151c20 con 0x7ff654156db0 2024-05-08T23:38:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.849+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff644006450 con 0x7ff654156db0 2024-05-08T23:38:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.849+0000 7ff63b7fe700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 0x7ff64006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.850+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff644088640 con 0x7ff654156db0 2024-05-08T23:38:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.850+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 0x7ff64006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.851+0000 7ff652ffd700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 0x7ff64006d220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fd0 tx=0x7ff64c009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.889+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6340052f0 con 0x7ff654156db0 2024-05-08T23:38:29.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:29.905+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff644017080 con 0x7ff654156db0 2024-05-08T23:38:30.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.612+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff634005b20 con 0x7ff654156db0 2024-05-08T23:38:30.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.617+0000 7ff63b7fe700 1 -- 172.21.15.33:0/1322462741 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ff644057250 con 0x7ff654156db0 2024-05-08T23:38:30.615 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":46,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:38:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.621+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 msgr2=0x7ff64006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.621+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 0x7ff64006d220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fd0 tx=0x7ff64c009000 comp rx=0 tx=0).stop 2024-05-08T23:38:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.622+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 msgr2=0x7ff654155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:30.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.622+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 0x7ff654155e10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff644009fd0 tx=0x7ff64400b6c0 comp rx=0 tx=0).stop 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 shutdown_connections 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff64006ad60 0x7ff64006d220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff654156db0 0x7ff654155e10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6540c70c0 0x7ff654155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 --2- 172.21.15.33:0/1322462741 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff654150510 0x7ff6541fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.623+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 >> 172.21.15.33:0/1322462741 conn(0x7ff6540c8360 msgr2=0x7ff6540c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.628+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 shutdown_connections 2024-05-08T23:38:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:30.634+0000 7ff6593ea700 1 -- 172.21.15.33:0/1322462741 wait complete. 2024-05-08T23:38:33.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:38:34.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.160+0000 7fdaa91bd700 1 Processor -- start 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.160+0000 7fdaa91bd700 1 -- start start 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.161+0000 7fdaa91bd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 0x7fdaa40c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.161+0000 7fdaa91bd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa406ba40 con 0x7fdaa41589d0 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.161+0000 7fdaa91bd700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa4070930 con 0x7fdaa40c2770 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.161+0000 7fdaa91bd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa415f180 con 0x7fdaa40714d0 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.163+0000 7fdaa2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdaa41589d0 0x7fdaa406a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49960/0 (socket says 172.21.15.33:49960) 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.163+0000 7fdaa2d9d700 1 -- 172.21.15.33:0/3896162642 learned_addr learned my addr 172.21.15.33:0/3896162642 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:34.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.173+0000 7fdaa359e700 1 --2- 172.21.15.33:0/3896162642 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 0x7fdaa40c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:34.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.175+0000 7fdaa359e700 1 -- 172.21.15.33:0/3896162642 >> v1:172.21.15.42:6789/0 conn(0x7fdaa40c2770 legacy=0x7fdaa4070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:34.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.175+0000 7fdaa359e700 1 -- 172.21.15.33:0/3896162642 >> v1:172.21.15.33:6789/0 conn(0x7fdaa41589d0 legacy=0x7fdaa406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:34.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.175+0000 7fdaa359e700 1 -- 172.21.15.33:0/3896162642 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa406b4c0 con 0x7fdaa40714d0 2024-05-08T23:38:34.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.177+0000 7fdaa359e700 1 --2- 172.21.15.33:0/3896162642 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 0x7fdaa40c21a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009dc0 tx=0x7fda9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832b3ae009a1f6d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.219+0000 7fdaa1d9b700 1 -- 172.21.15.33:0/3896162642 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00e070 con 0x7fdaa40714d0 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.219+0000 7fdaa1d9b700 1 -- 172.21.15.33:0/3896162642 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c00bad0 con 0x7fdaa40714d0 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.219+0000 7fdaa1d9b700 1 -- 172.21.15.33:0/3896162642 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00be80 con 0x7fdaa40714d0 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.220+0000 7fdaa91bd700 1 -- 172.21.15.33:0/3896162642 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 msgr2=0x7fdaa40c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.220+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/3896162642 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 0x7fdaa40c21a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009dc0 tx=0x7fda9c00b330 comp rx=0 tx=0).stop 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.221+0000 7fdaa91bd700 1 -- 172.21.15.33:0/3896162642 shutdown_connections 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.221+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/3896162642 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa40714d0 0x7fdaa40c21a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:34.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.221+0000 7fdaa91bd700 1 -- 172.21.15.33:0/3896162642 >> 172.21.15.33:0/3896162642 conn(0x7fdaa40c8360 msgr2=0x7fdaa40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:34.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.225+0000 7fdaa91bd700 1 -- 172.21.15.33:0/3896162642 shutdown_connections 2024-05-08T23:38:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.225+0000 7fdaa91bd700 1 -- 172.21.15.33:0/3896162642 wait complete. 2024-05-08T23:38:34.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.227+0000 7fdaa91bd700 1 Processor -- start 2024-05-08T23:38:34.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.228+0000 7fdaa91bd700 1 -- start start 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaa40714d0 0x7fdaa40c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 0x7fdaa41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4160280 con 0x7fdaa40c2770 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa415fc50 con 0x7fdaa40714d0 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.229+0000 7fdaa91bd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4148480 con 0x7fdaa41589d0 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41942/0 (socket says 172.21.15.33:41942) 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 -- 172.21.15.33:0/259852733 learned_addr learned my addr 172.21.15.33:0/259852733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 msgr2=0x7fdaa413d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 msgr2=0x7fdaa413d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.231+0000 7fdaa259c700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 0x7fdaa41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa2d9d700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaa40714d0 0x7fdaa40c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaa40714d0 msgr2=0x7fdaa40c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaa40714d0 0x7fdaa40c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 msgr2=0x7fdaa413d9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.232+0000 7fdaa359e700 1 -- 172.21.15.33:0/259852733 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009a70 con 0x7fdaa41589d0 2024-05-08T23:38:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.233+0000 7fdaa359e700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 0x7fdaa41422f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda9c015000 tx=0x7fda9c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:34.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.245+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0048d0 con 0x7fdaa41589d0 2024-05-08T23:38:34.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.245+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c012430 con 0x7fdaa41589d0 2024-05-08T23:38:34.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.245+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0127e0 con 0x7fdaa41589d0 2024-05-08T23:38:34.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.247+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa4148760 con 0x7fdaa41589d0 2024-05-08T23:38:34.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.247+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa415fea0 con 0x7fdaa41589d0 2024-05-08T23:38:34.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.253+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fda9c012940 con 0x7fdaa41589d0 2024-05-08T23:38:34.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.253+0000 7fda8b7fe700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 0x7fda9006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:34.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.253+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fda9c08ab40 con 0x7fdaa41589d0 2024-05-08T23:38:34.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.254+0000 7fdaa2d9d700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 0x7fda9006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:34.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.255+0000 7fdaa2d9d700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 0x7fda9006d220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fda98005910 tx=0x7fda98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:34.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.276+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda840052f0 con 0x7fdaa41589d0 2024-05-08T23:38:34.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:34.290+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fda9c055cc0 con 0x7fdaa41589d0 2024-05-08T23:38:35.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.103+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fda84005b20 con 0x7fdaa41589d0 2024-05-08T23:38:35.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.112+0000 7fda8b7fe700 1 -- 172.21.15.33:0/259852733 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fda9c021020 con 0x7fdaa41589d0 2024-05-08T23:38:35.118 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":46,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:38:35.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.125+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 msgr2=0x7fda9006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:35.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.125+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 0x7fda9006d220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fda98005910 tx=0x7fda98009040 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.125+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 msgr2=0x7fdaa41422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.125+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 0x7fdaa41422f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda9c015000 tx=0x7fda9c004ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 shutdown_connections 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fda9006ad60 0x7fda9006d220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdaa41589d0 0x7fdaa41422f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaa40714d0 0x7fdaa40c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 --2- 172.21.15.33:0/259852733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaa40c2770 0x7fdaa413d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:35.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 >> 172.21.15.33:0/259852733 conn(0x7fdaa40c8360 msgr2=0x7fdaa40ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.126+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 shutdown_connections 2024-05-08T23:38:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:35.127+0000 7fdaa91bd700 1 -- 172.21.15.33:0/259852733 wait complete. 2024-05-08T23:38:38.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.537+0000 7f7049448700 1 Processor -- start 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.537+0000 7f7049448700 1 -- start start 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704415f180 con 0x7f70440c2770 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704406ba40 con 0x7f704406a990 2024-05-08T23:38:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.538+0000 7f7049448700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044070930 con 0x7f70441589d0 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.543+0000 7f70427fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54210/0 (socket says 172.21.15.33:54210) 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 learned_addr learned my addr 172.21.15.33:0/110528709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.544+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.549+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.746+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.746+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.746+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.746+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.746+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.751+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.755+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.755+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.755+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:38.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:38.755+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:38:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.148+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.148+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.148+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.148+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.148+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:39.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.153+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.155+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:39.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.155+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.155+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.155+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:39.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.159+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.164+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.164+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.164+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.164+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:38:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.949+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.949+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.949+0000 7f70427fc700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.949+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.949+0000 7f70427fc700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.967+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.967+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.971+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:38:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.971+0000 7f7042ffd700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.971+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.971+0000 7f7042ffd700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.980+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:38:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.980+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:38:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.980+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:39.980+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f70440c21a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f70440c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 msgr2=0x7f704406ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.42:3300/0 conn(0x7f704406a990 0x7f704406ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 -- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 msgr2=0x7f70440c7d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 --2- 172.21.15.33:0/110528709 >> v2:172.21.15.33:3300/0 conn(0x7f70440c2770 0x7f70440c7d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 -- 172.21.15.33:0/110528709 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704406b4c0 con 0x7f702c0073d0 2024-05-08T23:38:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:41.539+0000 7f7040ff9700 1 -- 172.21.15.33:0/110528709 --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c00c140 con 0x7f702c003720 2024-05-08T23:38:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.516+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14439112 0 0) 0x7f702c00c140 con 0x7f702c003720 2024-05-08T23:38:42.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.516+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70300035b0 con 0x7f702c003720 2024-05-08T23:38:42.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691420825 0 0) 0x7f70300035b0 con 0x7f702c003720 2024-05-08T23:38:42.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 >> v1:172.21.15.33:6789/0 conn(0x7f702c0073d0 legacy=0x7f702c009890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:42.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c00d3a0 con 0x7f702c003720 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f703400e590 con 0x7f702c003720 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 >> v1:172.21.15.42:6789/0 conn(0x7f702c003720 legacy=0x7f702c005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f7030003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7030004190 0x7f7030004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f703000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703000bf50 con 0x7f7030004f90 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703000c090 con 0x7f7030004190 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703000c1d0 con 0x7f70441589d0 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.522+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 reap_dead start 2024-05-08T23:38:42.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f703000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 msgr2=0x7f7030003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f7030003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7030004190 msgr2=0x7f7030004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7030004190 0x7f7030004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 -- 172.21.15.33:0/110528709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c00d3a0 con 0x7f7030004f90 2024-05-08T23:38:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.523+0000 7f70437fe700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f703000ba10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7044067600 tx=0x7f7030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9304491030a2b0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.545+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0043e0 con 0x7f7030004f90 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.545+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c00f040 con 0x7f7030004f90 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.545+0000 7f7041ffb700 1 -- 172.21.15.33:0/110528709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c004780 con 0x7f7030004f90 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 -- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 msgr2=0x7f703000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f703000ba10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7044067600 tx=0x7f7030005880 comp rx=0 tx=0).stop 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 -- 172.21.15.33:0/110528709 shutdown_connections 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7030004190 0x7f7030004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f703000ba10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 --2- 172.21.15.33:0/110528709 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f70441589d0 0x7f7030003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 -- 172.21.15.33:0/110528709 >> 172.21.15.33:0/110528709 conn(0x7f70440c8360 msgr2=0x7f70440c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 -- 172.21.15.33:0/110528709 shutdown_connections 2024-05-08T23:38:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.548+0000 7f7049448700 1 -- 172.21.15.33:0/110528709 wait complete. 2024-05-08T23:38:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.550+0000 7f7049448700 1 Processor -- start 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.555+0000 7f7049448700 1 -- start start 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 0x7f704415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70441589d0 0x7f704415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704414a080 con 0x7f7030004f90 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044148b10 con 0x7f70441589d0 2024-05-08T23:38:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f7049448700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704413b4e0 con 0x7f7030004190 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f70427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f70427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54244/0 (socket says 172.21.15.33:54244) 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.557+0000 7f70427fc700 1 -- 172.21.15.33:0/423637446 learned_addr learned my addr 172.21.15.33:0/423637446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f70427fc700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 msgr2=0x7f704413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f70427fc700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 msgr2=0x7f704413b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f70427fc700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f70427fc700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:38:42.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f70437fe700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70441589d0 0x7f704415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:42.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.558+0000 7f7042ffd700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 0x7f704415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70441589d0 msgr2=0x7f704415b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70441589d0 0x7f704415b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 msgr2=0x7f704413b970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 -- 172.21.15.33:0/423637446 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044149a60 con 0x7f7030004190 2024-05-08T23:38:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.564+0000 7f7042ffd700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 0x7f704415aec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7034009100 tx=0x7f7034014a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.618+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034004440 con 0x7f7030004190 2024-05-08T23:38:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.618+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400c070 con 0x7f7030004190 2024-05-08T23:38:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.618+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70340038c0 con 0x7f7030004190 2024-05-08T23:38:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.619+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704414a290 con 0x7f7030004190 2024-05-08T23:38:42.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.619+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704413b120 con 0x7f7030004190 2024-05-08T23:38:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.623+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7034003a20 con 0x7f7030004190 2024-05-08T23:38:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.623+0000 7f702b7fe700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 0x7f703006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.624+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7034089e50 con 0x7f7030004190 2024-05-08T23:38:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.624+0000 7f70427fc700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 0x7f703006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.625+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704413b7c0 con 0x7f7030004190 2024-05-08T23:38:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.625+0000 7f70427fc700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 0x7f703006ea80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7044149360 tx=0x7f7038006e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:42.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:42.642+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7034058a90 con 0x7f7030004190 2024-05-08T23:38:43.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.498+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f70440c5630 con 0x7f7030004190 2024-05-08T23:38:43.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.499+0000 7f702b7fe700 1 -- 172.21.15.33:0/423637446 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f7034058620 con 0x7f7030004190 2024-05-08T23:38:43.504 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":54,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 msgr2=0x7f703006ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 0x7f703006ea80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7044149360 tx=0x7f7038006e10 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 msgr2=0x7f704415aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 0x7f704415aec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7034009100 tx=0x7f7034014a30 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 shutdown_connections 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f703006c5c0 0x7f703006ea80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70441589d0 0x7f704415b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7030004190 0x7f704415aec0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7034009100 tx=0x7f7034014a30 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 --2- 172.21.15.33:0/423637446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7030004f90 0x7f704413b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.508+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 >> 172.21.15.33:0/423637446 conn(0x7f70440c8360 msgr2=0x7f70440c4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:43.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.512+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 shutdown_connections 2024-05-08T23:38:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:43.516+0000 7f7049448700 1 -- 172.21.15.33:0/423637446 wait complete. 2024-05-08T23:38:43.571 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:38:43.571 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:38:43.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:38:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.077+0000 7ff2e0d34700 1 Processor -- start 2024-05-08T23:38:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.080+0000 7ff2e0d34700 1 -- start start 2024-05-08T23:38:44.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2e0d34700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff2dc0c9220 0x7ff2dc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2e0d34700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2e0d34700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0711c0 con 0x7ff2dc15b360 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2e0d34700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc06b0a0 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2e0d34700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc06e320 con 0x7ff2dc0c9220 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.082+0000 7ff2dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.083+0000 7ff2dad9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54248/0 (socket says 172.21.15.33:54248) 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.083+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1336747523 learned_addr learned my addr 172.21.15.33:0/1336747523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.084+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1336747523 >> v2:172.21.15.42:3300/0 conn(0x7ff2dc0c9220 msgr2=0x7ff2dc0729c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.084+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1336747523 >> v2:172.21.15.42:3300/0 conn(0x7ff2dc0c9220 0x7ff2dc0729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.084+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1336747523 >> v1:172.21.15.42:6789/0 conn(0x7ff2dc15b360 legacy=0x7ff2dc06a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.084+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1336747523 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc071710 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.084+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1336747523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc06fc30 tx=0x7ff2d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c123d7f92beb5dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.099+0000 7ff2d959a700 1 -- 172.21.15.33:0/1336747523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40092d0 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.099+0000 7ff2d959a700 1 -- 172.21.15.33:0/1336747523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d401f070 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.099+0000 7ff2d959a700 1 -- 172.21.15.33:0/1336747523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4003c60 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.110+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1336747523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b1ad0 msgr2=0x7ff2dc0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.110+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1336747523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc06fc30 tx=0x7ff2d400b330 comp rx=0 tx=0).stop 2024-05-08T23:38:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.120+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1336747523 shutdown_connections 2024-05-08T23:38:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.120+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1336747523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b1ad0 0x7ff2dc0b1f00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.120+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1336747523 >> v2:172.21.15.42:3300/0 conn(0x7ff2dc0c9220 0x7ff2dc0729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.120+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1336747523 >> 172.21.15.33:0/1336747523 conn(0x7ff2dc06d2b0 msgr2=0x7ff2dc06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.127+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1336747523 shutdown_connections 2024-05-08T23:38:44.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.135+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1336747523 wait complete. 2024-05-08T23:38:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.137+0000 7ff2e0d34700 1 Processor -- start 2024-05-08T23:38:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.139+0000 7ff2e0d34700 1 -- start start 2024-05-08T23:38:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2dc0b1ad0 0x7ff2dc0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0c9220 0x7ff2dc0c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2dc15b360 0x7ff2dc14a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c63a0 con 0x7ff2dc0c9220 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f00c0 con 0x7ff2dc0b1ad0 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.140+0000 7ff2e0d34700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0cb390 con 0x7ff2dc15b360 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.141+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2dc15b360 0x7ff2dc14a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.141+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2dc15b360 0x7ff2dc14a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:49902/0 (socket says 172.21.15.33:49902) 2024-05-08T23:38:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.141+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1069809879 learned_addr learned my addr 172.21.15.33:0/1069809879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.142+0000 7ff2d9d9b700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0c9220 0x7ff2dc0c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.143+0000 7ff2d9d9b700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2dc15b360 msgr2=0x7ff2dc14a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.143+0000 7ff2d9d9b700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2dc15b360 0x7ff2dc14a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.143+0000 7ff2d9d9b700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2dc0b1ad0 msgr2=0x7ff2dc0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.143+0000 7ff2d9d9b700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2dc0b1ad0 0x7ff2dc0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.143+0000 7ff2d9d9b700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d4009b30 con 0x7ff2dc0c9220 2024-05-08T23:38:44.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.144+0000 7ff2d9d9b700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0c9220 0x7ff2dc0c4050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006fd0 tx=0x7ff2d000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:44.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.145+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011830 con 0x7ff2dc0c9220 2024-05-08T23:38:44.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.145+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d00119b0 con 0x7ff2dc0c9220 2024-05-08T23:38:44.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.145+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0cb650 con 0x7ff2dc0c9220 2024-05-08T23:38:44.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.145+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011d60 con 0x7ff2dc0c9220 2024-05-08T23:38:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.165+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc1ef790 con 0x7ff2dc0c9220 2024-05-08T23:38:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.166+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff2d001a400 con 0x7ff2dc0c9220 2024-05-08T23:38:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.167+0000 7ff2caffd700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2bc06ace0 0x7ff2bc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.168+0000 7ff2da59c700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2bc06ace0 0x7ff2bc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.169+0000 7ff2da59c700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2bc06ace0 0x7ff2bc06d1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009770 tx=0x7ff2cc006cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.173+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff2d0057b50 con 0x7ff2dc0c9220 2024-05-08T23:38:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.182+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 0x7ff2dc1ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.182+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0749b0 con 0x7ff2dc1f0430 2024-05-08T23:38:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.183+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 0x7ff2dc1ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.183+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2dc06be10 con 0x7ff2dc1f0430 2024-05-08T23:38:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.185+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 0x7ff2dc1ef9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b370 tx=0x7ff2d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.189+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004230 con 0x7ff2dc1f0430 2024-05-08T23:38:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.191+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff2d400bf20 con 0x7ff2dc1f0430 2024-05-08T23:38:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.191+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 msgr2=0x7ff2dc1ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.191+0000 7ff2caffd700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 0x7ff2dc1ef9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b370 tx=0x7ff2d4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.191+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 msgr2=0x7ff2dc1ef9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.191+0000 7ff2caffd700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc1f0430 0x7ff2dc1ef9b0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400b370 tx=0x7ff2d4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.220+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 0x7ff2dc14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.220+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f0230 con 0x7ff2dc0caa50 2024-05-08T23:38:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.221+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 0x7ff2dc14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.222+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1069809879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2dc06c970 con 0x7ff2dc0caa50 2024-05-08T23:38:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.223+0000 7ff2dad9d700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 0x7ff2dc14a640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4037000 tx=0x7ff2d4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d403d780 con 0x7ff2dc0caa50 2024-05-08T23:38:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4459 (secure 0 0 0) 0x7ff2d4041070 con 0x7ff2dc0caa50 2024-05-08T23:38:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 msgr2=0x7ff2dc14a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 0x7ff2dc14a640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4037000 tx=0x7ff2d4022000 comp rx=0 tx=0).stop 2024-05-08T23:38:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 msgr2=0x7ff2dc14a640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.229+0000 7ff2caffd700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0caa50 0x7ff2dc14a640 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4037000 tx=0x7ff2d4022000 comp rx=0 tx=0).stop 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:44.230 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:44.232 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:44.233 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:44.234 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:44.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:44.240 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2bc06ace0 msgr2=0x7ff2bc06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff2bc06ace0 0x7ff2bc06d1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009770 tx=0x7ff2cc006cd0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0c9220 msgr2=0x7ff2dc0c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 --2- 172.21.15.33:0/1069809879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0c9220 0x7ff2dc0c4050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0006fd0 tx=0x7ff2d000b7e0 comp rx=0 tx=0).stop 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2dad9d700 1 -- 172.21.15.33:0/1069809879 reap_dead start 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 shutdown_connections 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.232+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 >> 172.21.15.33:0/1069809879 conn(0x7ff2dc06d2b0 msgr2=0x7ff2dc06d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.233+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 shutdown_connections 2024-05-08T23:38:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:44.233+0000 7ff2e0d34700 1 -- 172.21.15.33:0/1069809879 wait complete. 2024-05-08T23:38:44.268 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:38:44.268 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:38:44.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:38:45.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.120+0000 7f7ef7792700 1 Processor -- start 2024-05-08T23:38:45.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.121+0000 7f7ef7792700 1 -- start start 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.122+0000 7f7ef7792700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7ef00714d0 0x7f7ef00c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.122+0000 7f7ef7792700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef006ba40 con 0x7f7ef00c2770 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.122+0000 7f7ef7792700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef0070930 con 0x7f7ef01589d0 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.122+0000 7f7ef7792700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef015f180 con 0x7f7ef00714d0 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.127+0000 7f7ef4d2d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7ef00c2770 0x7f7ef0070dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40966/0 (socket says 172.21.15.33:40966) 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.127+0000 7f7ef4d2d700 1 -- 172.21.15.33:0/2918116505 learned_addr learned my addr 172.21.15.33:0/2918116505 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.127+0000 7f7ef5d2f700 1 --2- 172.21.15.33:0/2918116505 >> v2:172.21.15.33:3300/0 conn(0x7f7ef00714d0 0x7f7ef00c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.128+0000 7f7ef5d2f700 1 -- 172.21.15.33:0/2918116505 >> v1:172.21.15.42:6789/0 conn(0x7f7ef01589d0 legacy=0x7f7ef006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.128+0000 7f7ef5d2f700 1 -- 172.21.15.33:0/2918116505 >> v1:172.21.15.33:6789/0 conn(0x7f7ef00c2770 legacy=0x7f7ef0070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.128+0000 7f7ef5d2f700 1 -- 172.21.15.33:0/2918116505 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef006b4c0 con 0x7f7ef00714d0 2024-05-08T23:38:45.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.128+0000 7f7ef5d2f700 1 --2- 172.21.15.33:0/2918116505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00714d0 0x7f7ef00c21a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009840 tx=0x7f7ee0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bc77433783760b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.135+0000 7f7ee7fff700 1 -- 172.21.15.33:0/2918116505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0011830 con 0x7f7ef00714d0 2024-05-08T23:38:45.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.135+0000 7f7ee7fff700 1 -- 172.21.15.33:0/2918116505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee0011990 con 0x7f7ef00714d0 2024-05-08T23:38:45.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.135+0000 7f7ee7fff700 1 -- 172.21.15.33:0/2918116505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0011d20 con 0x7f7ef00714d0 2024-05-08T23:38:45.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.136+0000 7f7ef7792700 1 -- 172.21.15.33:0/2918116505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00714d0 msgr2=0x7f7ef00c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.136+0000 7f7ef7792700 1 --2- 172.21.15.33:0/2918116505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00714d0 0x7f7ef00c21a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009840 tx=0x7f7ee0009b50 comp rx=0 tx=0).stop 2024-05-08T23:38:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.137+0000 7f7ef7792700 1 -- 172.21.15.33:0/2918116505 shutdown_connections 2024-05-08T23:38:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.137+0000 7f7ef7792700 1 --2- 172.21.15.33:0/2918116505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00714d0 0x7f7ef00c21a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.137+0000 7f7ef7792700 1 -- 172.21.15.33:0/2918116505 >> 172.21.15.33:0/2918116505 conn(0x7f7ef00c8360 msgr2=0x7f7ef00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.137+0000 7f7ef7792700 1 -- 172.21.15.33:0/2918116505 shutdown_connections 2024-05-08T23:38:45.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.137+0000 7f7ef7792700 1 -- 172.21.15.33:0/2918116505 wait complete. 2024-05-08T23:38:45.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.139+0000 7f7ef7792700 1 Processor -- start 2024-05-08T23:38:45.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.142+0000 7f7ef7792700 1 -- start start 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef00714d0 0x7f7ef01590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 0x7f7ef01460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7ef01589d0 0x7f7ef00c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef015ad50 con 0x7f7ef00c2770 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef015a7d0 con 0x7f7ef00714d0 2024-05-08T23:38:45.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.143+0000 7f7ef7792700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef01478d0 con 0x7f7ef01589d0 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 0x7f7ef01460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 0x7f7ef01460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54306/0 (socket says 172.21.15.33:54306) 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 -- 172.21.15.33:0/4071425907 learned_addr learned my addr 172.21.15.33:0/4071425907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef552e700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef00714d0 0x7f7ef01590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7ef01589d0 msgr2=0x7f7ef00c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7ef01589d0 0x7f7ef00c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef00714d0 msgr2=0x7f7ef01590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef00714d0 0x7f7ef01590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.144+0000 7f7ef4d2d700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00094a0 con 0x7f7ef00c2770 2024-05-08T23:38:45.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.145+0000 7f7ef4d2d700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 0x7f7ef01460b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c550 tx=0x7f7eec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.146+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec005d70 con 0x7f7ef00c2770 2024-05-08T23:38:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.146+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0070d0 con 0x7f7ef00c2770 2024-05-08T23:38:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.146+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007440 con 0x7f7ef00c2770 2024-05-08T23:38:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.146+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef01465f0 con 0x7f7ef00c2770 2024-05-08T23:38:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.159+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7eec0075a0 con 0x7f7ef00c2770 2024-05-08T23:38:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.162+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef015bc80 con 0x7f7ef00c2770 2024-05-08T23:38:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.163+0000 7f7ef7792700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 0x7f7ef0147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.163+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0074a20 con 0x7f7ef015b170 2024-05-08T23:38:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.165+0000 7f7ee5ffb700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7ed406ac90 0x7f7ed406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.166+0000 7f7ef5d2f700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7ed406ac90 0x7f7ed406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.167+0000 7f7ef5d2f700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7ed406ac90 0x7f7ed406d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01584e0 tx=0x7f7ee0010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.168+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7eec0579a0 con 0x7f7ef00c2770 2024-05-08T23:38:45.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.173+0000 7f7ef552e700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 0x7f7ef0147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.174+0000 7f7ef552e700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ef015f650 con 0x7f7ef015b170 2024-05-08T23:38:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.185+0000 7f7ef552e700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 0x7f7ef0147b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80012e0 tx=0x7f7ee800d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003c10 con 0x7f7ef015b170 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7ee8003d70 con 0x7f7ef015b170 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 msgr2=0x7f7ef0147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 0x7f7ef0147b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80012e0 tx=0x7f7ee800d430 comp rx=0 tx=0).stop 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 msgr2=0x7f7ef0147b20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.187+0000 7f7ee5ffb700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef015b170 0x7f7ef0147b20 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80012e0 tx=0x7f7ee800d430 comp rx=0 tx=0).stop 2024-05-08T23:38:45.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.218+0000 7f7ef7792700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 0x7f7ef0140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.218+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef006b760 con 0x7f7ef0147250 2024-05-08T23:38:45.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.218+0000 7f7ef552e700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 0x7f7ef0140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.218+0000 7f7ef552e700 1 -- 172.21.15.33:0/4071425907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ef0158db0 con 0x7f7ef0147250 2024-05-08T23:38:45.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.220+0000 7f7ef552e700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 0x7f7ef0140e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8002000 tx=0x7f7ee80014a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.221+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8023070 con 0x7f7ef0147250 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.222+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4940 (secure 0 0 0) 0x7f7ee801ee70 con 0x7f7ef0147250 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.222+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 msgr2=0x7f7ef0140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.222+0000 7f7ee5ffb700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 0x7f7ef0140e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8002000 tx=0x7f7ee80014a0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.222+0000 7f7ee5ffb700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 msgr2=0x7f7ef0140e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:45.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.222+0000 7f7ee5ffb700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ef0147250 0x7f7ef0140e20 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8002000 tx=0x7f7ee80014a0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.223 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:45.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.225 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:45.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.233 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:45.234 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7ed406ac90 msgr2=0x7f7ed406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7ed406ac90 0x7f7ed406d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01584e0 tx=0x7f7ee0010000 comp rx=0 tx=0).stop 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 msgr2=0x7f7ef01460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 --2- 172.21.15.33:0/4071425907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef00c2770 0x7f7ef01460b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c550 tx=0x7f7eec00c860 comp rx=0 tx=0).stop 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef5d2f700 1 -- 172.21.15.33:0/4071425907 reap_dead start 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef5d2f700 1 -- 172.21.15.33:0/4071425907 reap_dead start 2024-05-08T23:38:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 shutdown_connections 2024-05-08T23:38:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.242+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 >> 172.21.15.33:0/4071425907 conn(0x7f7ef00c8360 msgr2=0x7f7ef00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.243+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 shutdown_connections 2024-05-08T23:38:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.243+0000 7f7ef7792700 1 -- 172.21.15.33:0/4071425907 wait complete. 2024-05-08T23:38:45.296 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:38:45.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:38:45.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:38:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.849+0000 7fef26bf0700 1 Processor -- start 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.849+0000 7fef26bf0700 1 -- start start 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef26bf0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef201589d0 0x7fef200c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef26bf0700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef200c2770 0x7fef200c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef26bf0700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2006ba40 con 0x7fef2006a990 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef26bf0700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2015f180 con 0x7fef200c2770 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef26bf0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20070930 con 0x7fef201589d0 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef2518d700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fef2006a990 0x7fef20070dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:40710/0 (socket says 172.21.15.33:40710) 2024-05-08T23:38:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.851+0000 7fef2518d700 1 -- 172.21.15.33:0/1687955986 learned_addr learned my addr 172.21.15.33:0/1687955986 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150262649 0 0) 0x7fef2006ba40 con 0x7fef2006a990 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef080035b0 con 0x7fef2006a990 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851895844 0 0) 0x7fef080035b0 con 0x7fef2006a990 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef201589d0 msgr2=0x7fef200c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef201589d0 0x7fef200c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 >> v2:172.21.15.33:3300/0 conn(0x7fef200c2770 msgr2=0x7fef200c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 --2- 172.21.15.33:0/1687955986 >> v2:172.21.15.33:3300/0 conn(0x7fef200c2770 0x7fef200c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2006b4c0 con 0x7fef2006a990 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fef10001670 con 0x7fef2006a990 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 >> v1:172.21.15.42:6789/0 conn(0x7fef2006a990 legacy=0x7fef20070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef08003a00 0x7fef08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef08004850 0x7fef08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef08005260 0x7fef0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.852+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef20070930 con 0x7fef08003a00 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0800c190 con 0x7fef08005260 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0800c350 con 0x7fef08004850 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2498c700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef08003a00 0x7fef08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef08005260 0x7fef0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 -- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef08004850 msgr2=0x7fef08004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef08004850 0x7fef08004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 -- 172.21.15.33:0/1687955986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef08003a00 msgr2=0x7fef08003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef08003a00 0x7fef08003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.853+0000 7fef2518d700 1 -- 172.21.15.33:0/1687955986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2006b4c0 con 0x7fef08005260 2024-05-08T23:38:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.855+0000 7fef2518d700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef08005260 0x7fef0800ba10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c2c0 tx=0x7fef08005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8454eee236cd069c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.887+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000ce20 con 0x7fef08005260 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.887+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10005960 con 0x7fef08005260 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.887+0000 7fef1f7fe700 1 -- 172.21.15.33:0/1687955986 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10005d10 con 0x7fef08005260 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 -- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef08005260 msgr2=0x7fef0800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 --2- 172.21.15.33:0/1687955986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef08005260 0x7fef0800ba10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c2c0 tx=0x7fef08005880 comp rx=0 tx=0).stop 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef2518d700 1 -- 172.21.15.33:0/1687955986 reap_dead start 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 -- 172.21.15.33:0/1687955986 shutdown_connections 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 -- 172.21.15.33:0/1687955986 >> 172.21.15.33:0/1687955986 conn(0x7fef200c8360 msgr2=0x7fef200c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 -- 172.21.15.33:0/1687955986 shutdown_connections 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.889+0000 7fef26bf0700 1 -- 172.21.15.33:0/1687955986 wait complete. 2024-05-08T23:38:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.891+0000 7fef26bf0700 1 Processor -- start 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.891+0000 7fef26bf0700 1 -- start start 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef2014a250 0x7fef2015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef2014a630 0x7fef20159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 0x7fef201433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef20159780 con 0x7fef2015a4d0 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015a0e0 con 0x7fef2014a250 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef26bf0700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20143c60 con 0x7fef2014a630 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef2518d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 0x7fef201433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef2518d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 0x7fef201433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52708/0 (socket says 172.21.15.33:52708) 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.893+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 learned_addr learned my addr 172.21.15.33:0/207257465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef2014a630 msgr2=0x7fef20159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef2014a630 0x7fef20159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef2014a250 msgr2=0x7fef2015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef2014a250 0x7fef2015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20143f40 con 0x7fef2015a4d0 2024-05-08T23:38:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.894+0000 7fef2518d700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 0x7fef201433b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef10011000 tx=0x7fef10001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.925+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10023070 con 0x7fef2015a4d0 2024-05-08T23:38:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.925+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef100072e0 con 0x7fef2015a4d0 2024-05-08T23:38:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.925+0000 7fef26bf0700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef20144220 con 0x7fef2015a4d0 2024-05-08T23:38:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.926+0000 7fef26bf0700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef201593a0 con 0x7fef2015a4d0 2024-05-08T23:38:45.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.926+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10001c30 con 0x7fef2015a4d0 2024-05-08T23:38:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.927+0000 7fef06ffd700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 0x7fef2013c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.927+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef20074a20 con 0x7fef202004f0 2024-05-08T23:38:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.928+0000 7fef2498c700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 0x7fef2013c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.930+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fef100074b0 con 0x7fef2015a4d0 2024-05-08T23:38:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.930+0000 7fef1d7fa700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fef0c06ad60 0x7fef0c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.930+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fef10089d70 con 0x7fef2015a4d0 2024-05-08T23:38:45.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.931+0000 7fef1ffff700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fef0c06ad60 0x7fef0c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.931+0000 7fef2498c700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef2015f650 con 0x7fef202004f0 2024-05-08T23:38:45.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.931+0000 7fef1ffff700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fef0c06ad60 0x7fef0c06d220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef20068370 tx=0x7fef14006d20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.932+0000 7fef2498c700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 0x7fef2013c6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef1800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800cb70 con 0x7fef202004f0 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fef1800ccd0 con 0x7fef202004f0 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 msgr2=0x7fef2013c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 0x7fef2013c6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef1800ef90 comp rx=0 tx=0).stop 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 msgr2=0x7fef2013c6f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:45.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.938+0000 7fef1d7fa700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202004f0 0x7fef2013c6f0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef1800ef90 comp rx=0 tx=0).stop 2024-05-08T23:38:45.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.955+0000 7fef06ffd700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 0x7fef2013cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:45.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.955+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef201443b0 con 0x7fef202008d0 2024-05-08T23:38:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.959+0000 7fef2498c700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 0x7fef2013cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.980+0000 7fef2498c700 1 -- 172.21.15.33:0/207257465 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef2006b760 con 0x7fef202008d0 2024-05-08T23:38:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.993+0000 7fef2498c700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 0x7fef2013cc30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef18009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 3, 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:45.993 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:45.994 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:45.996 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:45.997 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:46.000 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:46.001 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.002 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.003 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.004 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:46.005 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.993+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1801e8e0 con 0x7fef202008d0 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.994+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4967 (secure 0 0 0) 0x7fef1801ea40 con 0x7fef202008d0 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.994+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 msgr2=0x7fef2013cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.994+0000 7fef1d7fa700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 0x7fef2013cc30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef18009540 comp rx=0 tx=0).stop 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.994+0000 7fef1d7fa700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 msgr2=0x7fef2013cc30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.994+0000 7fef1d7fa700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fef202008d0 0x7fef2013cc30 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef20073190 tx=0x7fef18009540 comp rx=0 tx=0).stop 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fef0c06ad60 msgr2=0x7fef0c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef06ffd700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fef0c06ad60 0x7fef0c06d220 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef20068370 tx=0x7fef14006d20 comp rx=0 tx=0).stop 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 msgr2=0x7fef201433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef06ffd700 1 --2- 172.21.15.33:0/207257465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2015a4d0 0x7fef201433b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef10011000 tx=0x7fef10001930 comp rx=0 tx=0).stop 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 reap_dead start 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:45.999+0000 7fef2518d700 1 -- 172.21.15.33:0/207257465 reap_dead start 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.000+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 shutdown_connections 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.000+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 >> 172.21.15.33:0/207257465 conn(0x7fef200c8360 msgr2=0x7fef200cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.000+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 shutdown_connections 2024-05-08T23:38:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.000+0000 7fef06ffd700 1 -- 172.21.15.33:0/207257465 wait complete. 2024-05-08T23:38:46.035 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:38:46.035 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:38:46.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.438+0000 7f7c3f0e7700 1 Processor -- start 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.438+0000 7f7c3f0e7700 1 -- start start 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f7c381589d0 0x7f7c380c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c380c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3806a990 0x7f7c3806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3815f180 con 0x7f7c3806a990 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3806ba40 con 0x7f7c381589d0 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3f0e7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38070930 con 0x7f7c380c2770 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3d684700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3806a990 0x7f7c3806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3d684700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7c3806a990 0x7f7c3806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52714/0 (socket says 172.21.15.33:52714) 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.440+0000 7f7c3d684700 1 -- 172.21.15.33:0/3679408924 learned_addr learned my addr 172.21.15.33:0/3679408924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c37fff700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c380c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c3d684700 1 -- 172.21.15.33:0/3679408924 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 msgr2=0x7f7c380c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c3d684700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c380c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c3d684700 1 -- 172.21.15.33:0/3679408924 >> v2:172.21.15.42:3300/0 conn(0x7f7c381589d0 msgr2=0x7f7c380c21a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c3d684700 1 --2- 172.21.15.33:0/3679408924 >> v2:172.21.15.42:3300/0 conn(0x7f7c381589d0 0x7f7c380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.441+0000 7f7c3d684700 1 -- 172.21.15.33:0/3679408924 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3806b4c0 con 0x7f7c3806a990 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.442+0000 7f7c3d684700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c3806ae20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806f0c0 tx=0x7f7c3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc04fe7a07ad5f07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.442+0000 7f7c377fe700 1 -- 172.21.15.33:0/3679408924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000e070 con 0x7f7c3806a990 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.442+0000 7f7c377fe700 1 -- 172.21.15.33:0/3679408924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3000b9c0 con 0x7f7c3806a990 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.442+0000 7f7c377fe700 1 -- 172.21.15.33:0/3679408924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000bd70 con 0x7f7c3806a990 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.443+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/3679408924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 msgr2=0x7f7c3806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.443+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c3806ae20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806f0c0 tx=0x7f7c3000b330 comp rx=0 tx=0).stop 2024-05-08T23:38:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.466+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/3679408924 shutdown_connections 2024-05-08T23:38:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.466+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c3806ae20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.466+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/3679408924 >> v2:172.21.15.42:3300/0 conn(0x7f7c381589d0 0x7f7c380c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.466+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/3679408924 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c380c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.466+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/3679408924 >> 172.21.15.33:0/3679408924 conn(0x7f7c380c8360 msgr2=0x7f7c380c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.467+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/3679408924 shutdown_connections 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.467+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/3679408924 wait complete. 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.469+0000 7f7c3f0e7700 1 Processor -- start 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.470+0000 7f7c3f0e7700 1 -- start start 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c38155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c3814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38154310 con 0x7f7c3806a990 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3814ece0 con 0x7f7c381589d0 2024-05-08T23:38:46.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.471+0000 7f7c3f0e7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c38156720 con 0x7f7c380c2770 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.477+0000 7f7c3d684700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.477+0000 7f7c3d684700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:50360/0 (socket says 172.21.15.33:50360) 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.477+0000 7f7c3d684700 1 -- 172.21.15.33:0/466139824 learned_addr learned my addr 172.21.15.33:0/466139824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.481+0000 7f7c3d684700 1 -- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 msgr2=0x7f7c3814e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.481+0000 7f7c3d684700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c3814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.481+0000 7f7c3d684700 1 -- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 msgr2=0x7f7c38155000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.481+0000 7f7c3d684700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c38155000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.481+0000 7f7c3d684700 1 -- 172.21.15.33:0/466139824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009b30 con 0x7f7c381589d0 2024-05-08T23:38:46.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.492+0000 7f7c3d684700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806f5d0 tx=0x7f7c30004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:46.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.513+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004310 con 0x7f7c381589d0 2024-05-08T23:38:46.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.513+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30004470 con 0x7f7c381589d0 2024-05-08T23:38:46.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.513+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004800 con 0x7f7c381589d0 2024-05-08T23:38:46.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.513+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c38156ca0 con 0x7f7c381589d0 2024-05-08T23:38:46.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.513+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c381569c0 con 0x7f7c381589d0 2024-05-08T23:38:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.519+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f7c3001b430 con 0x7f7c381589d0 2024-05-08T23:38:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.519+0000 7f7c357fa700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 0x7f7c2406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:46.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.519+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7c3008a640 con 0x7f7c381589d0 2024-05-08T23:38:46.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.522+0000 7f7c3ce83700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 0x7f7c2406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:46.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.523+0000 7f7c3ce83700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 0x7f7c2406d1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009670 tx=0x7f7c28006bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:46.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.543+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c180052f0 con 0x7f7c381589d0 2024-05-08T23:38:46.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:46.568+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f7c30055730 con 0x7f7c381589d0 2024-05-08T23:38:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.332+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f7c18005db0 con 0x7f7c381589d0 2024-05-08T23:38:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.343+0000 7f7c357fa700 1 -- 172.21.15.33:0/466139824 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f7c3001f070 con 0x7f7c381589d0 2024-05-08T23:38:47.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 msgr2=0x7f7c2406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 0x7f7c2406d1d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009670 tx=0x7f7c28006bd0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 msgr2=0x7f7c381532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c3806f5d0 tx=0x7f7c30004ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 shutdown_connections 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f7c2406ad10 0x7f7c2406d1d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c381589d0 0x7f7c381532d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c3806a990 0x7f7c38155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 --2- 172.21.15.33:0/466139824 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7c380c2770 0x7f7c3814e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.346+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 >> 172.21.15.33:0/466139824 conn(0x7f7c380c8360 msgr2=0x7f7c38073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.347+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 shutdown_connections 2024-05-08T23:38:47.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:47.347+0000 7f7c3f0e7700 1 -- 172.21.15.33:0/466139824 wait complete. 2024-05-08T23:38:47.371 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:38:50.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:38:50.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:38:57.372 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:38:57.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:38:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.724+0000 7fd3b98b4700 1 Processor -- start 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.724+0000 7fd3b98b4700 1 -- start start 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.725+0000 7fd3b98b4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b40c2770 0x7fd3b40c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.725+0000 7fd3b98b4700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3b40c7d80 0x7fd3b4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.725+0000 7fd3b98b4700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b406ba40 con 0x7fd3b41589d0 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.725+0000 7fd3b98b4700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b415f180 con 0x7fd3b40c7d80 2024-05-08T23:38:57.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.725+0000 7fd3b98b4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b40c8210 con 0x7fd3b40c2770 2024-05-08T23:38:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.731+0000 7fd3b37fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3b40c7d80 0x7fd3b4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.731+0000 7fd3b37fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3b40c7d80 0x7fd3b4070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:34554/0 (socket says 172.21.15.33:34554) 2024-05-08T23:38:57.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.731+0000 7fd3b37fe700 1 -- 172.21.15.33:0/2785069041 learned_addr learned my addr 172.21.15.33:0/2785069041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.732+0000 7fd3b37fe700 1 -- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b40c2770 msgr2=0x7fd3b40c21a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:38:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.732+0000 7fd3b37fe700 1 --2- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b40c2770 0x7fd3b40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.732+0000 7fd3b37fe700 1 -- 172.21.15.33:0/2785069041 >> v1:172.21.15.42:6789/0 conn(0x7fd3b41589d0 legacy=0x7fd3b406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:38:57.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.732+0000 7fd3b37fe700 1 -- 172.21.15.33:0/2785069041 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4070930 con 0x7fd3b40c7d80 2024-05-08T23:38:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.733+0000 7fd3b37fe700 1 --2- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b4070dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e47f9a6b58f1066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.749+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/2785069041 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00e070 con 0x7fd3b40c7d80 2024-05-08T23:38:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.749+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/2785069041 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac003a20 con 0x7fd3b40c7d80 2024-05-08T23:38:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.749+0000 7fd3b1ffb700 1 -- 172.21.15.33:0/2785069041 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003dd0 con 0x7fd3b40c7d80 2024-05-08T23:38:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.750+0000 7fd3b98b4700 1 -- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 msgr2=0x7fd3b4070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.750+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b4070dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009e50 tx=0x7fd3ac00b330 comp rx=0 tx=0).stop 2024-05-08T23:38:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.757+0000 7fd3b98b4700 1 -- 172.21.15.33:0/2785069041 shutdown_connections 2024-05-08T23:38:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.757+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b4070dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.757+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/2785069041 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b40c2770 0x7fd3b40c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.757+0000 7fd3b98b4700 1 -- 172.21.15.33:0/2785069041 >> 172.21.15.33:0/2785069041 conn(0x7fd3b40c8360 msgr2=0x7fd3b40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:57.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.757+0000 7fd3b98b4700 1 -- 172.21.15.33:0/2785069041 shutdown_connections 2024-05-08T23:38:57.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.758+0000 7fd3b98b4700 1 -- 172.21.15.33:0/2785069041 wait complete. 2024-05-08T23:38:57.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.760+0000 7fd3b98b4700 1 Processor -- start 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.760+0000 7fd3b98b4700 1 -- start start 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b40c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b41589d0 0x7fd3b415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b415c070 con 0x7fd3b40c2770 2024-05-08T23:38:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b415afc0 con 0x7fd3b40c7d80 2024-05-08T23:38:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.761+0000 7fd3b98b4700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b4159780 con 0x7fd3b41589d0 2024-05-08T23:38:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.762+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.762+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34174/0 (socket says 172.21.15.33:34174) 2024-05-08T23:38:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.762+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1110223568 learned_addr learned my addr 172.21.15.33:0/1110223568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b41589d0 msgr2=0x7fd3b415b450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:38:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b41589d0 0x7fd3b415b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 msgr2=0x7fd3b40c32c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:38:57.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b40c32c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:57.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 -- 172.21.15.33:0/1110223568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac009b30 con 0x7fd3b40c2770 2024-05-08T23:38:57.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.763+0000 7fd3b2ffd700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b550 tx=0x7fd3a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.777+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4011830 con 0x7fd3b40c2770 2024-05-08T23:38:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.777+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a40119b0 con 0x7fd3b40c2770 2024-05-08T23:38:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.777+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4011d60 con 0x7fd3b40c2770 2024-05-08T23:38:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.777+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b4136010 con 0x7fd3b40c2770 2024-05-08T23:38:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.777+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b4135a50 con 0x7fd3b40c2770 2024-05-08T23:38:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.782+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd3a401a410 con 0x7fd3b40c2770 2024-05-08T23:38:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.783+0000 7fd39bfff700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 0x7fd39c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.783+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd3a4088f20 con 0x7fd3b40c2770 2024-05-08T23:38:57.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.784+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3940052f0 con 0x7fd3b40c2770 2024-05-08T23:38:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.785+0000 7fd3b27fc700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 0x7fd39c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:57.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.786+0000 7fd3b27fc700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 0x7fd39c06d3e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006fd0 tx=0x7fd3a8006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:57.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:57.798+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd3a4053f80 con 0x7fd3b40c2770 2024-05-08T23:38:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.653+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd394005b20 con 0x7fd3b40c2770 2024-05-08T23:38:58.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.654+0000 7fd39bfff700 1 -- 172.21.15.33:0/1110223568 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fd3a4015030 con 0x7fd3b40c2770 2024-05-08T23:38:58.656 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":54,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:38:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.700+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 msgr2=0x7fd39c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.700+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 0x7fd39c06d3e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006fd0 tx=0x7fd3a8006bc0 comp rx=0 tx=0).stop 2024-05-08T23:38:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.700+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 msgr2=0x7fd3b41350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.700+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b550 tx=0x7fd3a400b910 comp rx=0 tx=0).stop 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 shutdown_connections 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd39c06af20 0x7fd39c06d3e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3b41589d0 0x7fd3b415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b40c2770 0x7fd3b41350c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 --2- 172.21.15.33:0/1110223568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b40c7d80 0x7fd3b40c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.701+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 >> 172.21.15.33:0/1110223568 conn(0x7fd3b40c8360 msgr2=0x7fd3b414aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.702+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 shutdown_connections 2024-05-08T23:38:58.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:58.719+0000 7fd3b98b4700 1 -- 172.21.15.33:0/1110223568 wait complete. 2024-05-08T23:38:58.783 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:38:58.783 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:38:58.783 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:38:58.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:38:59.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.223+0000 7fccbe3cd700 1 Processor -- start 2024-05-08T23:38:59.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.223+0000 7fccbe3cd700 1 -- start start 2024-05-08T23:38:59.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.225+0000 7fccbe3cd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccb80c9050 0x7fccb8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.225+0000 7fccbe3cd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb8072e40 con 0x7fccb80c58c0 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.225+0000 7fccbe3cd700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb806b920 con 0x7fccb81670c0 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.225+0000 7fccbe3cd700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb806aa30 con 0x7fccb80c9050 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.228+0000 7fccb7fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fccb80c58c0 0x7fccb8070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54232/0 (socket says 172.21.15.33:54232) 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.228+0000 7fccb7fff700 1 -- 172.21.15.33:0/3489998114 learned_addr learned my addr 172.21.15.33:0/3489998114 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.232+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435734690 0 0) 0x7fccb8072e40 con 0x7fccb80c58c0 2024-05-08T23:38:59.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.235+0000 7fccb77fe700 1 --2- 172.21.15.33:0/3489998114 >> v2:172.21.15.33:3300/0 conn(0x7fccb80c9050 0x7fccb8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.263+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca40035b0 con 0x7fccb80c58c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.263+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522011261 0 0) 0x7fccb806b920 con 0x7fccb81670c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.263+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb8072e40 con 0x7fccb81670c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642136862 0 0) 0x7fcca40035b0 con 0x7fccb80c58c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 >> v2:172.21.15.33:3300/0 conn(0x7fccb80c9050 msgr2=0x7fccb8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 --2- 172.21.15.33:0/3489998114 >> v2:172.21.15.33:3300/0 conn(0x7fccb80c9050 0x7fccb8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 >> v1:172.21.15.42:6789/0 conn(0x7fccb81670c0 legacy=0x7fccb8171960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb806ad10 con 0x7fccb80c58c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fcca8001670 con 0x7fccb80c58c0 2024-05-08T23:38:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 >> v1:172.21.15.33:6789/0 conn(0x7fccb80c58c0 legacy=0x7fccb8070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcca4004c20 0x7fcca4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca40037f0 0x7fcca4003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 0x7fcca400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00015b0 con 0x7fcca4004280 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb806aa30 con 0x7fcca40037f0 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.266+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca400c2b0 con 0x7fcca4004c20 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.267+0000 7fccbc96a700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 0x7fcca400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccb7fff700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcca4004c20 0x7fcca4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 -- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcca4004c20 msgr2=0x7fcca4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcca4004c20 0x7fcca4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 -- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca40037f0 msgr2=0x7fcca4003c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca40037f0 0x7fcca4003c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 -- 172.21.15.33:0/3489998114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40035b0 con 0x7fcca4004280 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 -- 172.21.15.33:0/3489998114 reap_dead start 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.268+0000 7fccbc96a700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 0x7fcca400ba10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fccb0002130 tx=0x7fcca4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f136ef84d699ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.269+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb000cde0 con 0x7fcca4004280 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.269+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0003a00 con 0x7fcca4004280 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.269+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3489998114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0003d90 con 0x7fcca4004280 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.270+0000 7fccbe3cd700 1 -- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 msgr2=0x7fcca400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.270+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 0x7fcca400ba10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fccb0002130 tx=0x7fcca4005880 comp rx=0 tx=0).stop 2024-05-08T23:38:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.271+0000 7fccb77fe700 1 --2- 172.21.15.33:0/3489998114 >> v2:172.21.15.33:3300/0 conn(0x7fccb80c9050 0x7fccb8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:38:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.276+0000 7fccbe3cd700 1 -- 172.21.15.33:0/3489998114 shutdown_connections 2024-05-08T23:38:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.276+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/3489998114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca4004280 0x7fcca400ba10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.276+0000 7fccbe3cd700 1 -- 172.21.15.33:0/3489998114 >> 172.21.15.33:0/3489998114 conn(0x7fccb80c8360 msgr2=0x7fccb80c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.276+0000 7fccbe3cd700 1 -- 172.21.15.33:0/3489998114 shutdown_connections 2024-05-08T23:38:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.276+0000 7fccbe3cd700 1 -- 172.21.15.33:0/3489998114 wait complete. 2024-05-08T23:38:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.279+0000 7fccbe3cd700 1 Processor -- start 2024-05-08T23:38:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.279+0000 7fccbe3cd700 1 -- start start 2024-05-08T23:38:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4004280 0x7fccb8168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb816b040 0x7fccb8166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 0x7fccb8164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8169020 con 0x7fccb816b040 2024-05-08T23:38:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8158000 con 0x7fcca4004280 2024-05-08T23:38:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.280+0000 7fccbe3cd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81ffbd0 con 0x7fccb8200340 2024-05-08T23:38:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.282+0000 7fccbc96a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 0x7fccb8164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.282+0000 7fccbc96a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 0x7fccb8164d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:50204/0 (socket says 172.21.15.33:50204) 2024-05-08T23:38:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.282+0000 7fccbc96a700 1 -- 172.21.15.33:0/2265484591 learned_addr learned my addr 172.21.15.33:0/2265484591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.282+0000 7fccb77fe700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb816b040 0x7fccb8166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.287+0000 7fccb7fff700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4004280 0x7fccb8168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccb77fe700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 msgr2=0x7fccb8164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccb77fe700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 0x7fccb8164d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccb77fe700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4004280 msgr2=0x7fccb8168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccb77fe700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcca4004280 0x7fccb8168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccb77fe700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8169aa0 con 0x7fccb816b040 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.288+0000 7fccbc96a700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fccb8200340 0x7fccb8164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:38:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccb77fe700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb816b040 0x7fccb8166780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fccb806d510 tx=0x7fcca000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000e8b0 con 0x7fccb816b040 2024-05-08T23:38:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca000ea50 con 0x7fccb816b040 2024-05-08T23:38:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000ee00 con 0x7fccb816b040 2024-05-08T23:38:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb8158250 con 0x7fccb816b040 2024-05-08T23:38:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.290+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb8158810 con 0x7fccb816b040 2024-05-08T23:38:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.305+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcca000b040 con 0x7fccb816b040 2024-05-08T23:38:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.306+0000 7fccb4ff9700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc9806ad90 0x7fcc9806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.306+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fcca00893a0 con 0x7fccb816b040 2024-05-08T23:38:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.306+0000 7fccb7fff700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc9806ad90 0x7fcc9806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.308+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 0x7fccb8200720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.308+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8074a20 con 0x7fccb81ffd10 2024-05-08T23:38:59.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.308+0000 7fccbc96a700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 0x7fccb8200720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.309+0000 7fccbc96a700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccb806b5b0 con 0x7fccb81ffd10 2024-05-08T23:38:59.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccbc96a700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 0x7fccb8200720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccb816b6d0 tx=0x7fccb0001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00018e0 con 0x7fccb81ffd10 2024-05-08T23:38:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fccb003a070 con 0x7fccb81ffd10 2024-05-08T23:38:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 msgr2=0x7fccb8200720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 0x7fccb8200720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccb816b6d0 tx=0x7fccb0001ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 msgr2=0x7fccb8200720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.310+0000 7fccb4ff9700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb81ffd10 0x7fccb8200720 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccb816b6d0 tx=0x7fccb0001ad0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.312+0000 7fccb7fff700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc9806ad90 0x7fcc9806d250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcca8001b80 tx=0x7fcca8000ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.341+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 0x7fccb8165640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.341+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb806b2d0 con 0x7fccb8165260 2024-05-08T23:38:59.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.345+0000 7fccbc96a700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 0x7fccb8165640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.346+0000 7fccbc96a700 1 -- 172.21.15.33:0/2265484591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fccb80c7f90 con 0x7fccb8165260 2024-05-08T23:38:59.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.346+0000 7fccbc96a700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 0x7fccb8165640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccb0045000 tx=0x7fccb0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0035660 con 0x7fccb8165260 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4460 (secure 0 0 0) 0x7fccb00357c0 con 0x7fccb8165260 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 msgr2=0x7fccb8165640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 0x7fccb8165640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccb0045000 tx=0x7fccb0032000 comp rx=0 tx=0).stop 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 msgr2=0x7fccb8165640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:59.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.347+0000 7fccb4ff9700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb8165260 0x7fccb8165640 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccb0045000 tx=0x7fccb0032000 comp rx=0 tx=0).stop 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:59.354 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 16, 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.355 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.356 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:59.357 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.358 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.360 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.361 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.363 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.364 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.365 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:59.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:59.367 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc9806ad90 msgr2=0x7fcc9806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcc9806ad90 0x7fcc9806d250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcca8001b80 tx=0x7fcca8000ed0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb816b040 msgr2=0x7fccb8166780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbe3cd700 1 --2- 172.21.15.33:0/2265484591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccb816b040 0x7fccb8166780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fccb806d510 tx=0x7fcca000cc00 comp rx=0 tx=0).stop 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbc96a700 1 -- 172.21.15.33:0/2265484591 reap_dead start 2024-05-08T23:38:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.395+0000 7fccbc96a700 1 -- 172.21.15.33:0/2265484591 reap_dead start 2024-05-08T23:38:59.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.399+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 shutdown_connections 2024-05-08T23:38:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.399+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 >> 172.21.15.33:0/2265484591 conn(0x7fccb80c8360 msgr2=0x7fccb80c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.400+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 shutdown_connections 2024-05-08T23:38:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.400+0000 7fccbe3cd700 1 -- 172.21.15.33:0/2265484591 wait complete. 2024-05-08T23:38:59.432 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:38:59.432 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:38:59.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:38:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.820+0000 7f58307f3700 1 Processor -- start 2024-05-08T23:38:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.823+0000 7f58307f3700 1 -- start start 2024-05-08T23:38:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.825+0000 7f58307f3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5828154770 0x7f582815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.825+0000 7f58307f3700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58280c5c20 con 0x7f58280d00b0 2024-05-08T23:38:59.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.825+0000 7f58307f3700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5828154b50 con 0x7f58280c4c70 2024-05-08T23:38:59.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.825+0000 7f58307f3700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58280b30b0 con 0x7f5828154770 2024-05-08T23:38:59.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.825+0000 7f582ed90700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f58280d00b0 0x7f5828164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:54240/0 (socket says 172.21.15.33:54240) 2024-05-08T23:38:59.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.826+0000 7f582ed90700 1 -- 172.21.15.33:0/688339913 learned_addr learned my addr 172.21.15.33:0/688339913 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:38:59.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.827+0000 7f582e58f700 1 --2- 172.21.15.33:0/688339913 >> v2:172.21.15.33:3300/0 conn(0x7f5828154770 0x7f582815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.829+0000 7f582e58f700 1 -- 172.21.15.33:0/688339913 >> v1:172.21.15.42:6789/0 conn(0x7f58280c4c70 legacy=0x7f58280c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:38:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.829+0000 7f582e58f700 1 -- 172.21.15.33:0/688339913 >> v1:172.21.15.33:6789/0 conn(0x7f58280d00b0 legacy=0x7f5828164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.829+0000 7f582e58f700 1 -- 172.21.15.33:0/688339913 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280caed0 con 0x7f5828154770 2024-05-08T23:38:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.829+0000 7f582e58f700 1 --2- 172.21.15.33:0/688339913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f582815ae50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5818009840 tx=0x7f5818009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718ff1d6fc5ea9e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.831+0000 7f582d58d700 1 -- 172.21.15.33:0/688339913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004230 con 0x7f5828154770 2024-05-08T23:38:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.831+0000 7f582d58d700 1 -- 172.21.15.33:0/688339913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818004390 con 0x7f5828154770 2024-05-08T23:38:59.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.831+0000 7f582d58d700 1 -- 172.21.15.33:0/688339913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58180036a0 con 0x7f5828154770 2024-05-08T23:38:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.832+0000 7f58307f3700 1 -- 172.21.15.33:0/688339913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 msgr2=0x7f582815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.832+0000 7f58307f3700 1 --2- 172.21.15.33:0/688339913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f582815ae50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5818009840 tx=0x7f5818009b50 comp rx=0 tx=0).stop 2024-05-08T23:38:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.833+0000 7f58307f3700 1 -- 172.21.15.33:0/688339913 shutdown_connections 2024-05-08T23:38:59.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.833+0000 7f58307f3700 1 --2- 172.21.15.33:0/688339913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f582815ae50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.833+0000 7f58307f3700 1 -- 172.21.15.33:0/688339913 >> 172.21.15.33:0/688339913 conn(0x7f5828066ae0 msgr2=0x7f5828066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.833+0000 7f58307f3700 1 -- 172.21.15.33:0/688339913 shutdown_connections 2024-05-08T23:38:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.833+0000 7f58307f3700 1 -- 172.21.15.33:0/688339913 wait complete. 2024-05-08T23:38:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.835+0000 7f58307f3700 1 Processor -- start 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.836+0000 7f58307f3700 1 -- start start 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280c4c70 0x7f5828148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58280d00b0 0x7f582814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f58280b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582814b490 con 0x7f5828154770 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280cefb0 con 0x7f58280c4c70 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f58307f3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582814b6c0 con 0x7f58280d00b0 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f582ed90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f58280b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f582ed90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f58280b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34236/0 (socket says 172.21.15.33:34236) 2024-05-08T23:38:59.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.837+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 learned_addr learned my addr 172.21.15.33:0/1120708448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.838+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58280d00b0 msgr2=0x7f582814d780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.838+0000 7f582ed90700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58280d00b0 0x7f582814d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.838+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280c4c70 msgr2=0x7f5828148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.838+0000 7f582ed90700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280c4c70 0x7f5828148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.839+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180094a0 con 0x7f5828154770 2024-05-08T23:38:59.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.839+0000 7f582ed90700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f58280b3c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5820002000 tx=0x7f582000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.841+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ad40 con 0x7f5828154770 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.841+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820001e30 con 0x7f5828154770 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.841+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000aea0 con 0x7f5828154770 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.842+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582814b980 con 0x7f5828154770 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.842+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58280cec40 con 0x7f5828154770 2024-05-08T23:38:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.843+0000 7f58307f3700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 0x7f58280b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.843+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828074690 con 0x7f58280b59b0 2024-05-08T23:38:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.845+0000 7f582e58f700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 0x7f58280b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.845+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5820007310 con 0x7f5828154770 2024-05-08T23:38:59.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.845+0000 7f582e58f700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58280c52f0 con 0x7f58280b59b0 2024-05-08T23:38:59.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.846+0000 7f582e58f700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 0x7f58280b41a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5818009840 tx=0x7f5818003980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.846+0000 7f581effd700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f581406ae70 0x7f581406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.846+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f5820089350 con 0x7f5828154770 2024-05-08T23:38:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f582dd8e700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f581406ae70 0x7f581406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818011630 con 0x7f58280b59b0 2024-05-08T23:38:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5818011790 con 0x7f58280b59b0 2024-05-08T23:38:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 msgr2=0x7f58280b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 0x7f58280b41a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5818009840 tx=0x7f5818003980 comp rx=0 tx=0).stop 2024-05-08T23:38:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 msgr2=0x7f58280b41a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.847+0000 7f581effd700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280b59b0 0x7f58280b41a0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5818009840 tx=0x7f5818003980 comp rx=0 tx=0).stop 2024-05-08T23:38:59.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.848+0000 7f582dd8e700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f581406ae70 0x7f581406d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5828071f00 tx=0x7f5824007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.889+0000 7f58307f3700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 0x7f58280b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:38:59.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.889+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280cabf0 con 0x7f58280cf340 2024-05-08T23:38:59.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.891+0000 7f582e58f700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 0x7f58280b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:38:59.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.895+0000 7f582e58f700 1 -- 172.21.15.33:0/1120708448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f58280c55d0 con 0x7f58280cf340 2024-05-08T23:38:59.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.896+0000 7f582e58f700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 0x7f58280b46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5818009b90 tx=0x7f5818011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.897+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818029b20 con 0x7f58280cf340 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.898+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4941 (secure 0 0 0) 0x7f5818024470 con 0x7f58280cf340 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.898+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 msgr2=0x7f58280b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.898+0000 7f581effd700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 0x7f58280b46e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5818009b90 tx=0x7f5818011000 comp rx=0 tx=0).stop 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.898+0000 7f581effd700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 msgr2=0x7f58280b46e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:38:59.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.898+0000 7f581effd700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58280cf340 0x7f58280b46e0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5818009b90 tx=0x7f5818011000 comp rx=0 tx=0).stop 2024-05-08T23:38:59.897 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:38:59.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:59.897 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 17, 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.898 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:38:59.899 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:38:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:38:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.904 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.905 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:38:59.907 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:38:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:38:59.910 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:38:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.905+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f581406ae70 msgr2=0x7f581406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.905+0000 7f58307f3700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f581406ae70 0x7f581406d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5828071f00 tx=0x7f5824007a20 comp rx=0 tx=0).stop 2024-05-08T23:38:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.905+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 msgr2=0x7f58280b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:38:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.905+0000 7f58307f3700 1 --2- 172.21.15.33:0/1120708448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5828154770 0x7f58280b3c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5820002000 tx=0x7f582000a340 comp rx=0 tx=0).stop 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.908+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 reap_dead start 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.908+0000 7f582ed90700 1 -- 172.21.15.33:0/1120708448 reap_dead start 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.908+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 shutdown_connections 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.908+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 >> 172.21.15.33:0/1120708448 conn(0x7f5828066ae0 msgr2=0x7f582814f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.908+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 shutdown_connections 2024-05-08T23:38:59.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:38:59.909+0000 7f58307f3700 1 -- 172.21.15.33:0/1120708448 wait complete. 2024-05-08T23:38:59.942 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:38:59.942 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:38:59.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:39:00.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.223+0000 7f8cae79e700 1 Processor -- start 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.224+0000 7f8cae79e700 1 -- start start 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.225+0000 7f8cae79e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8ca8150790 0x7f8ca80c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.225+0000 7f8cae79e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8071640 0x7f8ca8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.225+0000 7f8cae79e700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80c6eb0 con 0x7f8ca80c9430 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.225+0000 7f8cae79e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca8071ff0 con 0x7f8ca8150790 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.225+0000 7f8cae79e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81574e0 con 0x7f8ca8071640 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.226+0000 7f8cacf9b700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f8ca80c9430 0x7f8ca806e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:44970/0 (socket says 172.21.15.33:44970) 2024-05-08T23:39:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.226+0000 7f8cacf9b700 1 -- 172.21.15.33:0/2204533690 learned_addr learned my addr 172.21.15.33:0/2204533690 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 --2- 172.21.15.33:0/2204533690 >> v2:172.21.15.33:3300/0 conn(0x7f8ca8150790 0x7f8ca80c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 -- 172.21.15.33:0/2204533690 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8071640 msgr2=0x7f8ca8071ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 --2- 172.21.15.33:0/2204533690 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8071640 0x7f8ca8071ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 -- 172.21.15.33:0/2204533690 >> v1:172.21.15.42:6789/0 conn(0x7f8ca80c9430 legacy=0x7f8ca806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 -- 172.21.15.33:0/2204533690 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8156680 con 0x7f8ca8150790 2024-05-08T23:39:00.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.227+0000 7f8cad79c700 1 --2- 172.21.15.33:0/2204533690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca8150790 0x7f8ca80c6970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009840 tx=0x7f8c9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70d436593552462 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.228+0000 7f8c9bfff700 1 -- 172.21.15.33:0/2204533690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011830 con 0x7f8ca8150790 2024-05-08T23:39:00.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.228+0000 7f8c9bfff700 1 -- 172.21.15.33:0/2204533690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c011990 con 0x7f8ca8150790 2024-05-08T23:39:00.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.228+0000 7f8c9bfff700 1 -- 172.21.15.33:0/2204533690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011d20 con 0x7f8ca8150790 2024-05-08T23:39:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.232+0000 7f8cae79e700 1 -- 172.21.15.33:0/2204533690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca8150790 msgr2=0x7f8ca80c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.232+0000 7f8cae79e700 1 --2- 172.21.15.33:0/2204533690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca8150790 0x7f8ca80c6970 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009840 tx=0x7f8c9c009b50 comp rx=0 tx=0).stop 2024-05-08T23:39:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.233+0000 7f8cae79e700 1 -- 172.21.15.33:0/2204533690 shutdown_connections 2024-05-08T23:39:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.233+0000 7f8cae79e700 1 --2- 172.21.15.33:0/2204533690 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8071640 0x7f8ca8071ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.233+0000 7f8cae79e700 1 --2- 172.21.15.33:0/2204533690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca8150790 0x7f8ca80c6970 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.233+0000 7f8cae79e700 1 -- 172.21.15.33:0/2204533690 >> 172.21.15.33:0/2204533690 conn(0x7f8ca806c520 msgr2=0x7f8ca806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:00.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.239+0000 7f8cae79e700 1 -- 172.21.15.33:0/2204533690 shutdown_connections 2024-05-08T23:39:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.245+0000 7f8cae79e700 1 -- 172.21.15.33:0/2204533690 wait complete. 2024-05-08T23:39:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.247+0000 7f8cae79e700 1 Processor -- start 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.248+0000 7f8cae79e700 1 -- start start 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 0x7f8ca8146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca80c9430 0x7f8ca80c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8150790 0x7f8ca81dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80c5710 con 0x7f8ca80c9430 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8147230 con 0x7f8ca8071640 2024-05-08T23:39:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.249+0000 7f8cae79e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80c6070 con 0x7f8ca8150790 2024-05-08T23:39:00.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.250+0000 7f8cad79c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 0x7f8ca8146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.250+0000 7f8cad79c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 0x7f8ca8146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:34626/0 (socket says 172.21.15.33:34626) 2024-05-08T23:39:00.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.250+0000 7f8cad79c700 1 -- 172.21.15.33:0/1743427952 learned_addr learned my addr 172.21.15.33:0/1743427952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:00.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.253+0000 7f8cacf9b700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca80c9430 0x7f8ca80c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.254+0000 7f8cad79c700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8150790 msgr2=0x7f8ca81dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.254+0000 7f8cad79c700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8ca8150790 0x7f8ca81dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.254+0000 7f8cad79c700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca80c9430 msgr2=0x7f8ca80c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.254+0000 7f8cad79c700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ca80c9430 0x7f8ca80c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.254+0000 7f8cad79c700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0094a0 con 0x7f8ca8071640 2024-05-08T23:39:00.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.255+0000 7f8cad79c700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 0x7f8ca8146cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009f50 tx=0x7f8c9c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.256+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0038a0 con 0x7f8ca8071640 2024-05-08T23:39:00.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.256+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c01f070 con 0x7f8ca8071640 2024-05-08T23:39:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.256+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c003c60 con 0x7f8ca8071640 2024-05-08T23:39:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.256+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca81e9170 con 0x7f8ca8071640 2024-05-08T23:39:00.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.257+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca81e9730 con 0x7f8ca8071640 2024-05-08T23:39:00.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.259+0000 7f8cae79e700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 0x7f8c88004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.259+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88007f20 con 0x7f8c88002a40 2024-05-08T23:39:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.261+0000 7f8cacf9b700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 0x7f8c88004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.262+0000 7f8cacf9b700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c880080e0 con 0x7f8c88002a40 2024-05-08T23:39:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.263+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8c9c01a430 con 0x7f8ca8071640 2024-05-08T23:39:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.264+0000 7f8c99ffb700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8c8c06ac90 0x7f8c8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.264+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8c9c021070 con 0x7f8ca8071640 2024-05-08T23:39:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.264+0000 7f8cadf9d700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8c8c06ac90 0x7f8c8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.266+0000 7f8cacf9b700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 0x7f8c88004f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c2c20 tx=0x7f8ca400ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.268+0000 7f8cadf9d700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8c8c06ac90 0x7f8c8c06d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c50d0 tx=0x7f8ca00015b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4005b50 con 0x7f8c88002a40 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8ca4005cb0 con 0x7f8c88002a40 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 msgr2=0x7f8c88004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 0x7f8c88004f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c2c20 tx=0x7f8ca400ea90 comp rx=0 tx=0).stop 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 msgr2=0x7f8c88004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.274+0000 7f8c99ffb700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88002a40 0x7f8c88004f00 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c2c20 tx=0x7f8ca400ea90 comp rx=0 tx=0).stop 2024-05-08T23:39:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.295+0000 7f8cae79e700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 0x7f8c88007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.295+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8800a3e0 con 0x7f8c88006fc0 2024-05-08T23:39:00.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.299+0000 7f8cacf9b700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 0x7f8c88007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.300+0000 7f8cacf9b700 1 -- 172.21.15.33:0/1743427952 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8c8800a5a0 con 0x7f8c88006fc0 2024-05-08T23:39:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.300+0000 7f8cacf9b700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 0x7f8c88007450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4016000 tx=0x7f8ca4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.301+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4021070 con 0x7f8c88006fc0 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.307+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4968 (secure 0 0 0) 0x7f8ca401c6d0 con 0x7f8c88006fc0 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.307+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 msgr2=0x7f8c88007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.307+0000 7f8c99ffb700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 0x7f8c88007450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4016000 tx=0x7f8ca4005880 comp rx=0 tx=0).stop 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.307+0000 7f8c99ffb700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 msgr2=0x7f8c88007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.307+0000 7f8c99ffb700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8c88006fc0 0x7f8c88007450 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4016000 tx=0x7f8ca4005880 comp rx=0 tx=0).stop 2024-05-08T23:39:00.305 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 54, 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 17, 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:00.306 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:00.307 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:00.308 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:00.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:00.311 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:00.311 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:00.311 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:00.311 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.315 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.316 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:00.317 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:00.318 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:00.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:00.320 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:00.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:00.320 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:00.320 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8c8c06ac90 msgr2=0x7f8c8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8c8c06ac90 0x7f8c8c06d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c50d0 tx=0x7f8ca00015b0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 msgr2=0x7f8ca8146cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 --2- 172.21.15.33:0/1743427952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca8071640 0x7f8ca8146cf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009f50 tx=0x7f8c9c004750 comp rx=0 tx=0).stop 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cadf9d700 1 -- 172.21.15.33:0/1743427952 reap_dead start 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cadf9d700 1 -- 172.21.15.33:0/1743427952 reap_dead start 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 shutdown_connections 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.310+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 >> 172.21.15.33:0/1743427952 conn(0x7f8ca806c520 msgr2=0x7f8ca806dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.312+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 shutdown_connections 2024-05-08T23:39:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.312+0000 7f8cae79e700 1 -- 172.21.15.33:0/1743427952 wait complete. 2024-05-08T23:39:00.335 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:39:00.335 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-08T23:39:00.335 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:39:00.335 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-08T23:39:00.335 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:39:00.336 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:39:00.336 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:39:00.355 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:39:00.355 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:39:00.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.679+0000 7f4fad213700 1 Processor -- start 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.679+0000 7f4fad213700 1 -- start start 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fad213700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 0x7f4fa806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fad213700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4fa8156f50 0x7f4fa8070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fad213700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa806afb0 con 0x7f4fa80c6260 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fad213700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa8071790 con 0x7f4fa8150710 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fad213700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa8072880 con 0x7f4fa8156f50 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fa6d9d700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 msgr2=0x7f4fa806a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fa6d9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 0x7f4fa806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fa659c700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f4fa80c6260 0x7f4fa8071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:44980/0 (socket says 172.21.15.33:44980) 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fa659c700 1 -- 172.21.15.33:0/343312734 learned_addr learned my addr 172.21.15.33:0/343312734 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:00.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.681+0000 7f4fa759e700 1 --2- 172.21.15.33:0/343312734 >> v2:172.21.15.42:3300/0 conn(0x7f4fa8156f50 0x7f4fa8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa759e700 1 -- 172.21.15.33:0/343312734 >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 msgr2=0x7f4fa806a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa5d9b700 1 -- 172.21.15.33:0/343312734 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056732223 0 0) 0x7f4fa806afb0 con 0x7f4fa80c6260 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa759e700 1 --2- 172.21.15.33:0/343312734 >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 0x7f4fa806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa759e700 1 -- 172.21.15.33:0/343312734 >> v1:172.21.15.42:6789/0 conn(0x7f4fa80c6260 legacy=0x7f4fa8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa759e700 1 -- 172.21.15.33:0/343312734 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa806b550 con 0x7f4fa8156f50 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.683+0000 7f4fa759e700 1 --2- 172.21.15.33:0/343312734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa8070460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fa806ffa0 tx=0x7f4fa000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e24792b608db03d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.684+0000 7f4fa5d9b700 1 -- 172.21.15.33:0/343312734 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000e070 con 0x7f4fa8156f50 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.684+0000 7f4fa5d9b700 1 -- 172.21.15.33:0/343312734 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa000bbd0 con 0x7f4fa8156f50 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.684+0000 7f4fa5d9b700 1 -- 172.21.15.33:0/343312734 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0012430 con 0x7f4fa8156f50 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.684+0000 7f4fad213700 1 -- 172.21.15.33:0/343312734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 msgr2=0x7f4fa8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:00.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.684+0000 7f4fad213700 1 --2- 172.21.15.33:0/343312734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa8070460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fa806ffa0 tx=0x7f4fa000b330 comp rx=0 tx=0).stop 2024-05-08T23:39:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.693+0000 7f4fad213700 1 -- 172.21.15.33:0/343312734 shutdown_connections 2024-05-08T23:39:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.693+0000 7f4fad213700 1 --2- 172.21.15.33:0/343312734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa8070460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.693+0000 7f4fad213700 1 --2- 172.21.15.33:0/343312734 >> v2:172.21.15.33:3300/0 conn(0x7f4fa8150710 0x7f4fa806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.693+0000 7f4fad213700 1 -- 172.21.15.33:0/343312734 >> 172.21.15.33:0/343312734 conn(0x7f4fa80c8360 msgr2=0x7f4fa80c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.694+0000 7f4fad213700 1 -- 172.21.15.33:0/343312734 shutdown_connections 2024-05-08T23:39:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.695+0000 7f4fad213700 1 -- 172.21.15.33:0/343312734 wait complete. 2024-05-08T23:39:00.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.697+0000 7f4fad213700 1 Processor -- start 2024-05-08T23:39:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.699+0000 7f4fad213700 1 -- start start 2024-05-08T23:39:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 0x7f4fa8154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa81513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa813d780 con 0x7f4fa80c6260 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8155840 con 0x7f4fa8156f50 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fad213700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8155b20 con 0x7f4fa8150710 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fa6d9d700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 msgr2=0x7f4fa8154e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.700+0000 7f4fa6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 0x7f4fa8154e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:00.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:50248/0 (socket says 172.21.15.33:50248) 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 -- 172.21.15.33:0/2549664048 learned_addr learned my addr 172.21.15.33:0/2549664048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 -- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 msgr2=0x7f4fa81513d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa81513d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 -- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 msgr2=0x7f4fa8154e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 0x7f4fa8154e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:00.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 -- 172.21.15.33:0/2549664048 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009a70 con 0x7f4fa8150710 2024-05-08T23:39:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.705+0000 7f4fa659c700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c460 tx=0x7f4f9800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.706+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004c80 con 0x7f4fa8150710 2024-05-08T23:39:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.706+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98004e00 con 0x7f4fa8150710 2024-05-08T23:39:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.706+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa813d9d0 con 0x7f4fa8150710 2024-05-08T23:39:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.706+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98007310 con 0x7f4fa8150710 2024-05-08T23:39:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.706+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa8150eb0 con 0x7f4fa8150710 2024-05-08T23:39:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.707+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa8075460 con 0x7f4fa8150710 2024-05-08T23:39:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.708+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4f98007470 con 0x7f4fa8150710 2024-05-08T23:39:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.709+0000 7f4f977fe700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 0x7f4f8c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.709+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f4f98088fa0 con 0x7f4fa8150710 2024-05-08T23:39:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.709+0000 7f4fa6d9d700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 0x7f4f8c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.710+0000 7f4fa6d9d700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 0x7f4f8c06d100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c006fd0 tx=0x7f4f9c006dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:00.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:00.723+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4f980bc900 con 0x7f4fa8150710 2024-05-08T23:39:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.349+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4fa813ea60 con 0x7f4fa8150710 2024-05-08T23:39:01.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.355+0000 7f4f977fe700 1 -- 172.21.15.33:0/2549664048 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f4f98054140 con 0x7f4fa8150710 2024-05-08T23:39:01.353 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":54,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 msgr2=0x7f4f8c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 0x7f4f8c06d100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c006fd0 tx=0x7f4f9c006dc0 comp rx=0 tx=0).stop 2024-05-08T23:39:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 msgr2=0x7f4fa80ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c460 tx=0x7f4f9800c820 comp rx=0 tx=0).stop 2024-05-08T23:39:01.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 shutdown_connections 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4f8c06ac40 0x7f4f8c06d100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fa8156f50 0x7f4fa81513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fa80c6260 0x7f4fa8154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 --2- 172.21.15.33:0/2549664048 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4fa8150710 0x7f4fa80ca5e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.363+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 >> 172.21.15.33:0/2549664048 conn(0x7f4fa80c8360 msgr2=0x7f4fa814f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.364+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 shutdown_connections 2024-05-08T23:39:01.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:01.364+0000 7f4fad213700 1 -- 172.21.15.33:0/2549664048 wait complete. 2024-05-08T23:39:04.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.598+0000 7f08e618b700 1 Processor -- start 2024-05-08T23:39:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.598+0000 7f08e618b700 1 -- start start 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08e618b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f08e00c7370 0x7f08e0157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08e618b700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e0157610 con 0x7f08e01513d0 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08e618b700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e0157fe0 con 0x7f08e00c4160 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08e618b700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e00c3b10 con 0x7f08e00c7370 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08dffff700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f08e00c7370 msgr2=0x7f08e0157aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:04.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08df7fe700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f08e01513d0 legacy=0x7f08e015a5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:39:04.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.599+0000 7f08dffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f08e00c7370 0x7f08e0157aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08deffd700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f08e00c4160 0x7f08e0070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:44992/0 (socket says 172.21.15.33:44992) 2024-05-08T23:39:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08deffd700 1 -- 172.21.15.33:0/1880191962 learned_addr learned my addr 172.21.15.33:0/1880191962 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799832789 0 0) 0x7f08e0157fe0 con 0x7f08e00c4160 2024-05-08T23:39:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c80035b0 con 0x7f08e00c4160 2024-05-08T23:39:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 477641277 0 0) 0x7f08c80035b0 con 0x7f08e00c4160 2024-05-08T23:39:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 >> v2:172.21.15.33:3300/0 conn(0x7f08e00c7370 msgr2=0x7f08e0157aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 --2- 172.21.15.33:0/1880191962 >> v2:172.21.15.33:3300/0 conn(0x7f08e00c7370 0x7f08e0157aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 >> v1:172.21.15.33:6789/0 conn(0x7f08e01513d0 legacy=0x7f08e015a5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0156fe0 con 0x7f08e00c4160 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f08d4001670 con 0x7f08e00c4160 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.603+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 >> v1:172.21.15.42:6789/0 conn(0x7f08e00c4160 legacy=0x7f08e0070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08c8003a00 0x7f08c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 0x7f08c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8005330 0x7f08c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800c200 con 0x7f08c8005330 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800c370 con 0x7f08c8004850 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800c530 con 0x7f08c8003a00 2024-05-08T23:39:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08dffff700 1 -- 172.21.15.33:0/1880191962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8005330 msgr2=0x7f08c800ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08dffff700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8005330 0x7f08c800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 0x7f08c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 -- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08c8003a00 msgr2=0x7f08c8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08c8003a00 0x7f08c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 -- 172.21.15.33:0/1880191962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8005330 msgr2=0x7f08c800ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8005330 0x7f08c800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08deffd700 1 -- 172.21.15.33:0/1880191962 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80035b0 con 0x7f08c8004850 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.604+0000 7f08dffff700 1 -- 172.21.15.33:0/1880191962 reap_dead start 2024-05-08T23:39:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.605+0000 7f08deffd700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 0x7f08c8004cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08d4001eb0 tx=0x7f08c8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2719d13f83c0c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.606+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4007b70 con 0x7f08c8004850 2024-05-08T23:39:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.606+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d400cca0 con 0x7f08c8004850 2024-05-08T23:39:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.606+0000 7f08de7fc700 1 -- 172.21.15.33:0/1880191962 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4005ba0 con 0x7f08c8004850 2024-05-08T23:39:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.607+0000 7f08e618b700 1 -- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 msgr2=0x7f08c8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:04.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.607+0000 7f08e618b700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 0x7f08c8004cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08d4001eb0 tx=0x7f08c8005880 comp rx=0 tx=0).stop 2024-05-08T23:39:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.608+0000 7f08e618b700 1 -- 172.21.15.33:0/1880191962 shutdown_connections 2024-05-08T23:39:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.608+0000 7f08e618b700 1 --2- 172.21.15.33:0/1880191962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08c8004850 0x7f08c8004cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.608+0000 7f08e618b700 1 -- 172.21.15.33:0/1880191962 >> 172.21.15.33:0/1880191962 conn(0x7f08e00c8360 msgr2=0x7f08e00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.608+0000 7f08e618b700 1 -- 172.21.15.33:0/1880191962 shutdown_connections 2024-05-08T23:39:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.608+0000 7f08e618b700 1 -- 172.21.15.33:0/1880191962 wait complete. 2024-05-08T23:39:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.609+0000 7f08e618b700 1 Processor -- start 2024-05-08T23:39:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.610+0000 7f08e618b700 1 -- start start 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 0x7f08e014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08e014a180 0x7f08e0159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e014e1f0 con 0x7f08c8004850 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e01f41a0 con 0x7f08e014f530 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08e618b700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e014f0a0 con 0x7f08e014a180 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08df7fe700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 msgr2=0x7f08e014fd70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 0x7f08e014fd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08dffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08dffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:34672/0 (socket says 172.21.15.33:34672) 2024-05-08T23:39:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.611+0000 7f08dffff700 1 -- 172.21.15.33:0/1507296742 learned_addr learned my addr 172.21.15.33:0/1507296742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 -- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08e014a180 msgr2=0x7f08e0159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08e014a180 0x7f08e0159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 -- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 msgr2=0x7f08e014fd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 0x7f08e014fd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 -- 172.21.15.33:0/1507296742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e015a110 con 0x7f08e014f530 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08dffff700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08e006ffc0 tx=0x7f08d8009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.612+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d800cb50 con 0x7f08e014f530 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.613+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d8006e90 con 0x7f08e014f530 2024-05-08T23:39:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.613+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8016430 con 0x7f08e014f530 2024-05-08T23:39:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.613+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e014ea90 con 0x7f08e014f530 2024-05-08T23:39:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.613+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e01f3ec0 con 0x7f08e014f530 2024-05-08T23:39:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.615+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e014e8f0 con 0x7f08e014f530 2024-05-08T23:39:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.615+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f08d8012020 con 0x7f08e014f530 2024-05-08T23:39:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.616+0000 7f08c7fff700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 0x7f08cc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.616+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f08d8088e70 con 0x7f08e014f530 2024-05-08T23:39:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.616+0000 7f08df7fe700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 0x7f08cc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.616+0000 7f08df7fe700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 0x7f08cc06d100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f08d0003940 tx=0x7f08d0007c30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:04.630+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f08e014e8f0 con 0x7f08e014f530 2024-05-08T23:39:04.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.002+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f08e014e8f0 con 0x7f08e014f530 2024-05-08T23:39:05.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.003+0000 7f08c7fff700 1 -- 172.21.15.33:0/1507296742 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f08e014e8f0 con 0x7f08e014f530 2024-05-08T23:39:05.000 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":54,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:05.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 msgr2=0x7f08cc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 0x7f08cc06d100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f08d0003940 tx=0x7f08d0007c30 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 msgr2=0x7f08e01f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08e006ffc0 tx=0x7f08d8009e10 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 shutdown_connections 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f08cc06ac40 0x7f08cc06d100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08e014f530 0x7f08e01f3230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08c8004850 0x7f08e014fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 --2- 172.21.15.33:0/1507296742 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f08e014a180 0x7f08e0159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.005+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 >> 172.21.15.33:0/1507296742 conn(0x7f08e00c8360 msgr2=0x7f08e01561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.006+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 shutdown_connections 2024-05-08T23:39:05.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:05.006+0000 7f08e618b700 1 -- 172.21.15.33:0/1507296742 wait complete. 2024-05-08T23:39:08.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.413+0000 7f89e97f7700 1 Processor -- start 2024-05-08T23:39:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.414+0000 7f89e97f7700 1 -- start start 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.415+0000 7f89e97f7700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89e40c3a90 0x7f89e4158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e97f7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 0x7f89e4158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e97f7700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40c81b0 con 0x7f89e40c7dd0 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e97f7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e4072ed0 con 0x7f89e40c8e50 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e97f7700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40c2d60 con 0x7f89e40c3a90 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e2ffd700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f89e40c7dd0 legacy=0x7f89e406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e37fe700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 msgr2=0x7f89e4158750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.416+0000 7f89e37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 0x7f89e4158750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.419+0000 7f89e27fc700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89e40c3a90 0x7f89e4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.419+0000 7f89e27fc700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f89e40c3a90 0x7f89e4158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46210/0 (socket says 172.21.15.33:46210) 2024-05-08T23:39:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.419+0000 7f89e27fc700 1 -- 172.21.15.33:0/1848733442 learned_addr learned my addr 172.21.15.33:0/1848733442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:08.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.420+0000 7f89e27fc700 1 -- 172.21.15.33:0/1848733442 >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 msgr2=0x7f89e4158750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.420+0000 7f89e27fc700 1 --2- 172.21.15.33:0/1848733442 >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 0x7f89e4158750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.420+0000 7f89e27fc700 1 -- 172.21.15.33:0/1848733442 >> v1:172.21.15.33:6789/0 conn(0x7f89e40c7dd0 legacy=0x7f89e406a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.420+0000 7f89e27fc700 1 -- 172.21.15.33:0/1848733442 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4158d30 con 0x7f89e40c3a90 2024-05-08T23:39:08.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.426+0000 7f89e27fc700 1 --2- 172.21.15.33:0/1848733442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c3a90 0x7f89e4158210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89e4067400 tx=0x7f89d800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40bb27867b8babd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.433+0000 7f89e1ffb700 1 -- 172.21.15.33:0/1848733442 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8009b60 con 0x7f89e40c3a90 2024-05-08T23:39:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.434+0000 7f89e1ffb700 1 -- 172.21.15.33:0/1848733442 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d800c070 con 0x7f89e40c3a90 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.434+0000 7f89e1ffb700 1 -- 172.21.15.33:0/1848733442 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d800f360 con 0x7f89e40c3a90 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.434+0000 7f89e97f7700 1 -- 172.21.15.33:0/1848733442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c3a90 msgr2=0x7f89e4158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.434+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1848733442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c3a90 0x7f89e4158210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89e4067400 tx=0x7f89d800fc90 comp rx=0 tx=0).stop 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.435+0000 7f89e97f7700 1 -- 172.21.15.33:0/1848733442 shutdown_connections 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.435+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1848733442 >> v2:172.21.15.33:3300/0 conn(0x7f89e40c8e50 0x7f89e4158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.435+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1848733442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c3a90 0x7f89e4158210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.435+0000 7f89e97f7700 1 -- 172.21.15.33:0/1848733442 >> 172.21.15.33:0/1848733442 conn(0x7f89e40c8360 msgr2=0x7f89e40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.436+0000 7f89e97f7700 1 -- 172.21.15.33:0/1848733442 shutdown_connections 2024-05-08T23:39:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.436+0000 7f89e97f7700 1 -- 172.21.15.33:0/1848733442 wait complete. 2024-05-08T23:39:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.438+0000 7f89e97f7700 1 Processor -- start 2024-05-08T23:39:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.439+0000 7f89e97f7700 1 -- start start 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 0x7f89e4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f89e40c7dd0 0x7f89e41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41563e0 con 0x7f89e40c3a90 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41575b0 con 0x7f89e40c8e50 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.440+0000 7f89e97f7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4152c20 con 0x7f89e40c7dd0 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.441+0000 7f89e2ffd700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 msgr2=0x7f89e4157d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.441+0000 7f89e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 0x7f89e4157d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.444+0000 7f89e37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.444+0000 7f89e37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46216/0 (socket says 172.21.15.33:46216) 2024-05-08T23:39:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.444+0000 7f89e37fe700 1 -- 172.21.15.33:0/1109536658 learned_addr learned my addr 172.21.15.33:0/1109536658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.445+0000 7f89e27fc700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f89e40c7dd0 0x7f89e41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.449+0000 7f89e37fe700 1 -- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f89e40c7dd0 msgr2=0x7f89e41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.449+0000 7f89e37fe700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f89e40c7dd0 0x7f89e41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.449+0000 7f89e37fe700 1 -- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 msgr2=0x7f89e4157d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.449+0000 7f89e37fe700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 0x7f89e4157d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.449+0000 7f89e37fe700 1 -- 172.21.15.33:0/1109536658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d800f610 con 0x7f89e40c8e50 2024-05-08T23:39:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.457+0000 7f89e37fe700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bb80 tx=0x7f89dc00bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.457+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00cb60 con 0x7f89e40c8e50 2024-05-08T23:39:08.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.457+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00cce0 con 0x7f89e40c8e50 2024-05-08T23:39:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.457+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0156b0 con 0x7f89e40c8e50 2024-05-08T23:39:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.458+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e4152f00 con 0x7f89e40c8e50 2024-05-08T23:39:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.458+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e4157850 con 0x7f89e40c8e50 2024-05-08T23:39:08.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.460+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e4153420 con 0x7f89e40c8e50 2024-05-08T23:39:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.460+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f89dc012070 con 0x7f89e40c8e50 2024-05-08T23:39:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.461+0000 7f89cb7fe700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.461+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f89dc089340 con 0x7f89e40c8e50 2024-05-08T23:39:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.461+0000 7f89e2ffd700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.461+0000 7f89e2ffd700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 0x7f89cc06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4006dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:08.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:08.473+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f89dc054540 con 0x7f89e40c8e50 2024-05-08T23:39:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.326+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f89e4157a40 con 0x7f89e40c8e50 2024-05-08T23:39:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.332+0000 7f89cb7fe700 1 -- 172.21.15.33:0/1109536658 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f89dc057b60 con 0x7f89e40c8e50 2024-05-08T23:39:09.338 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":54,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:09.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.344+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 msgr2=0x7f89cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:09.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.344+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 0x7f89cc06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f89d4006fd0 tx=0x7f89d4006dc0 comp rx=0 tx=0).stop 2024-05-08T23:39:09.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.344+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 msgr2=0x7f89e414b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.344+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bb80 tx=0x7f89dc00bf40 comp rx=0 tx=0).stop 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 shutdown_connections 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f89cc06ae40 0x7f89cc06d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89e40c8e50 0x7f89e414b910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89e40c3a90 0x7f89e4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 --2- 172.21.15.33:0/1109536658 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f89e40c7dd0 0x7f89e41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.345+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 >> 172.21.15.33:0/1109536658 conn(0x7f89e40c8360 msgr2=0x7f89e4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.363+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 shutdown_connections 2024-05-08T23:39:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:09.363+0000 7f89e97f7700 1 -- 172.21.15.33:0/1109536658 wait complete. 2024-05-08T23:39:12.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.891+0000 7fa823176700 1 Processor -- start 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.892+0000 7fa823176700 1 -- start start 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.892+0000 7fa823176700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.892+0000 7fa823176700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa823176700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c1572f0 con 0x7fa81c0c70c0 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa823176700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c157c30 con 0x7fa81c150510 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa823176700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0c8110 con 0x7fa81c1577a0 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa820f12700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 msgr2=0x7fa81c0c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa820f12700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa821713700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa81bfff700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7fa81c0c70c0 0x7fa81c06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:44616/0 (socket says 172.21.15.33:44616) 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa81bfff700 1 -- 172.21.15.33:0/3377811003 learned_addr learned my addr 172.21.15.33:0/3377811003 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:12.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:12.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:12.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:12.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:12.893+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:13.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.094+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 msgr2=0x7fa81c0c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:13.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.094+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.097+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.113+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.113+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.113+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.113+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:13.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.495+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 msgr2=0x7fa81c0c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:13.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.495+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.519+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.520+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.520+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.520+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:13.520+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.298+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 msgr2=0x7fa81c0c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.298+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.323+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.327+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.327+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.327+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:14.327+0000 7fa821713700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:15.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.307+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265451734 0 0) 0x7fa808004ae0 con 0x7fa81c0c70c0 2024-05-08T23:39:15.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.307+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa80c0035b0 con 0x7fa81c0c70c0 2024-05-08T23:39:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053375128 0 0) 0x7fa80c0035b0 con 0x7fa81c0c70c0 2024-05-08T23:39:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 msgr2=0x7fa81c156db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa81c1577a0 0x7fa81c156db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 msgr2=0x7fa81c0c7780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 --2- 172.21.15.33:0/3377811003 >> v2:172.21.15.33:3300/0 conn(0x7fa81c150510 0x7fa81c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0c7e30 con 0x7fa81c0c70c0 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fa8080010d0 con 0x7fa81c0c70c0 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 >> v1:172.21.15.42:6789/0 conn(0x7fa81c0c70c0 legacy=0x7fa81c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa80c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c004850 0x7fa80c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa80c005210 0x7fa80c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c157c30 con 0x7fa80c004850 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0c8110 con 0x7fa80c005210 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.310+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c00c030 con 0x7fa80c003a00 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.311+0000 7fa81bfff700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c004850 msgr2=0x7fa80c004cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.311+0000 7fa81bfff700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c004850 0x7fa80c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.311+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa80c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:15.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.313+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa80c005210 msgr2=0x7fa80c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:15.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.313+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa80c005210 0x7fa80c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.313+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c004850 msgr2=0x7fa80c004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:15.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.313+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c004850 0x7fa80c004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.313+0000 7fa820f12700 1 -- 172.21.15.33:0/3377811003 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0c7e30 con 0x7fa80c003a00 2024-05-08T23:39:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.319+0000 7fa820f12700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa80c003e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa810009d50 tx=0x7fa80c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff15064d7e8b474 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.329+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810009ea0 con 0x7fa80c003a00 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.329+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000bd60 con 0x7fa80c003a00 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.329+0000 7fa81b7fe700 1 -- 172.21.15.33:0/3377811003 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8100098d0 con 0x7fa80c003a00 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.330+0000 7fa823176700 1 -- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 msgr2=0x7fa80c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.330+0000 7fa823176700 1 --2- 172.21.15.33:0/3377811003 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa80c003e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa810009d50 tx=0x7fa80c005880 comp rx=0 tx=0).stop 2024-05-08T23:39:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.330+0000 7fa821713700 1 -- 172.21.15.33:0/3377811003 reap_dead start 2024-05-08T23:39:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.336+0000 7fa823176700 1 -- 172.21.15.33:0/3377811003 shutdown_connections 2024-05-08T23:39:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.336+0000 7fa823176700 1 -- 172.21.15.33:0/3377811003 >> 172.21.15.33:0/3377811003 conn(0x7fa81c0c8360 msgr2=0x7fa81c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.336+0000 7fa823176700 1 -- 172.21.15.33:0/3377811003 shutdown_connections 2024-05-08T23:39:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.337+0000 7fa823176700 1 -- 172.21.15.33:0/3377811003 wait complete. 2024-05-08T23:39:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.339+0000 7fa823176700 1 Processor -- start 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.339+0000 7fa823176700 1 -- start start 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa81c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 0x7fa81c1ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1ef800 con 0x7fa81c1eee10 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c153920 con 0x7fa81c150b60 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa823176700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c14ce10 con 0x7fa80c003a00 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa821713700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 msgr2=0x7fa81c1ef2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.341+0000 7fa821713700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 0x7fa81c1ef2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.343+0000 7fa81bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.343+0000 7fa81bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46230/0 (socket says 172.21.15.33:46230) 2024-05-08T23:39:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.343+0000 7fa81bfff700 1 -- 172.21.15.33:0/2509290579 learned_addr learned my addr 172.21.15.33:0/2509290579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 -- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 msgr2=0x7fa81c1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa81c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 -- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 msgr2=0x7fa81c1ef2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 0x7fa81c1ef2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 -- 172.21.15.33:0/2509290579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0ca1b0 con 0x7fa81c150b60 2024-05-08T23:39:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.347+0000 7fa81bfff700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa808004c40 tx=0x7fa80800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.359+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808009040 con 0x7fa81c150b60 2024-05-08T23:39:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.359+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80800bae0 con 0x7fa81c150b60 2024-05-08T23:39:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.359+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80800be90 con 0x7fa81c150b60 2024-05-08T23:39:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.361+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c153b90 con 0x7fa81c150b60 2024-05-08T23:39:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.361+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c14d490 con 0x7fa81c150b60 2024-05-08T23:39:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.363+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c075460 con 0x7fa81c150b60 2024-05-08T23:39:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.381+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa80801b460 con 0x7fa81c150b60 2024-05-08T23:39:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.382+0000 7fa8197fa700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 0x7fa80006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.382+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa80808a4c0 con 0x7fa81c150b60 2024-05-08T23:39:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.382+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa80808a880 con 0x7fa81c150b60 2024-05-08T23:39:15.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.383+0000 7fa820f12700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 0x7fa80006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:15.384+0000 7fa820f12700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 0x7fa80006d1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa810000ea0 tx=0x7fa81000f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.133+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa81c1572f0 con 0x7fa81c150b60 2024-05-08T23:39:16.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.134+0000 7fa8197fa700 1 -- 172.21.15.33:0/2509290579 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fa81c1572f0 con 0x7fa81c150b60 2024-05-08T23:39:16.159 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":56,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.196+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 msgr2=0x7fa80006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.196+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 0x7fa80006d1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa810000ea0 tx=0x7fa81000f000 comp rx=0 tx=0).stop 2024-05-08T23:39:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.196+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 msgr2=0x7fa81c06ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.196+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa808004c40 tx=0x7fa80800dc90 comp rx=0 tx=0).stop 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 shutdown_connections 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa80006ad10 0x7fa80006d1d0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa810000ea0 tx=0x7fa81000f000 comp rx=0 tx=0).stop 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81c1eee10 0x7fa81c1ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa80c003a00 0x7fa81c1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 --2- 172.21.15.33:0/2509290579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa81c150b60 0x7fa81c06ac00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.201+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 >> 172.21.15.33:0/2509290579 conn(0x7fa81c0c8360 msgr2=0x7fa81c158040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.206+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 shutdown_connections 2024-05-08T23:39:16.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.212+0000 7fa823176700 1 -- 172.21.15.33:0/2509290579 wait complete. 2024-05-08T23:39:16.259 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:39:16.259 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:39:16.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:39:16.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.592+0000 7f73b6c46700 1 Processor -- start 2024-05-08T23:39:16.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.593+0000 7f73b6c46700 1 -- start start 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b6c46700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 0x7f73b0070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b6c46700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b6c46700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b0070010 con 0x7f73b00c2180 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b6c46700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b0070870 con 0x7f73b00c8e50 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b6c46700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0071020 con 0x7f73b006b660 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b49e2700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 msgr2=0x7f73b0070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.594+0000 7f73b49e2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 0x7f73b0070ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:16.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.595+0000 7f73b51e3700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7f73b00c2180 legacy=0x7f73b0071270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:39:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.596+0000 7f73affff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.596+0000 7f73affff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:43484/0 (socket says 172.21.15.33:43484) 2024-05-08T23:39:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.596+0000 7f73affff700 1 -- 172.21.15.33:0/3143031162 learned_addr learned my addr 172.21.15.33:0/3143031162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.597+0000 7f73affff700 1 -- 172.21.15.33:0/3143031162 >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 msgr2=0x7f73b0070ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.597+0000 7f73affff700 1 --2- 172.21.15.33:0/3143031162 >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 0x7f73b0070ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.597+0000 7f73affff700 1 -- 172.21.15.33:0/3143031162 >> v1:172.21.15.33:6789/0 conn(0x7f73b00c2180 legacy=0x7f73b0071270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.597+0000 7f73affff700 1 -- 172.21.15.33:0/3143031162 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b015a820 con 0x7f73b006b660 2024-05-08T23:39:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.597+0000 7f73affff700 1 --2- 172.21.15.33:0/3143031162 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f739c0098f0 tx=0x7f739c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b11e243ba0207a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.598+0000 7f73af7fe700 1 -- 172.21.15.33:0/3143031162 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004230 con 0x7f73b006b660 2024-05-08T23:39:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.598+0000 7f73af7fe700 1 -- 172.21.15.33:0/3143031162 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c004390 con 0x7f73b006b660 2024-05-08T23:39:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.598+0000 7f73af7fe700 1 -- 172.21.15.33:0/3143031162 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0036a0 con 0x7f73b006b660 2024-05-08T23:39:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 -- 172.21.15.33:0/3143031162 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 msgr2=0x7f73b006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 --2- 172.21.15.33:0/3143031162 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f739c0098f0 tx=0x7f739c009c00 comp rx=0 tx=0).stop 2024-05-08T23:39:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 -- 172.21.15.33:0/3143031162 shutdown_connections 2024-05-08T23:39:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 --2- 172.21.15.33:0/3143031162 >> v2:172.21.15.33:3300/0 conn(0x7f73b00c8e50 0x7f73b0070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 --2- 172.21.15.33:0/3143031162 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b006a990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.599+0000 7f73b6c46700 1 -- 172.21.15.33:0/3143031162 >> 172.21.15.33:0/3143031162 conn(0x7f73b00c8360 msgr2=0x7f73b00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:16.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.602+0000 7f73b6c46700 1 -- 172.21.15.33:0/3143031162 shutdown_connections 2024-05-08T23:39:16.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.613+0000 7f73b6c46700 1 -- 172.21.15.33:0/3143031162 wait complete. 2024-05-08T23:39:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.615+0000 7f73b6c46700 1 Processor -- start 2024-05-08T23:39:16.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.616+0000 7f73b6c46700 1 -- start start 2024-05-08T23:39:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b0144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b00c2180 0x7f73b0145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b00c8e50 0x7f73b013dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0141c90 con 0x7f73b00c2180 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0143540 con 0x7f73b00c8e50 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.617+0000 7f73b6c46700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0150a20 con 0x7f73b006b660 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.618+0000 7f73b51e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b00c8e50 0x7f73b013dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.618+0000 7f73b49e2700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b0144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.618+0000 7f73b51e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b00c8e50 0x7f73b013dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:42822/0 (socket says 172.21.15.33:42822) 2024-05-08T23:39:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.618+0000 7f73b51e3700 1 -- 172.21.15.33:0/1738642255 learned_addr learned my addr 172.21.15.33:0/1738642255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:16.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b00c8e50 msgr2=0x7f73b013dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b00c8e50 0x7f73b013dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b00c2180 msgr2=0x7f73b0145190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b00c2180 0x7f73b0145190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009550 con 0x7f73b006b660 2024-05-08T23:39:16.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.622+0000 7f73b49e2700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b0144c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73a400b550 tx=0x7f73a400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.630+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003e50 con 0x7f73b006b660 2024-05-08T23:39:16.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.630+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4005070 con 0x7f73b006b660 2024-05-08T23:39:16.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.630+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4005420 con 0x7f73b006b660 2024-05-08T23:39:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.631+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b01458c0 con 0x7f73b006b660 2024-05-08T23:39:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.631+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0143260 con 0x7f73b006b660 2024-05-08T23:39:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.632+0000 7f73b6c46700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 0x7f73b013e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.632+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0074a20 con 0x7f73b006b220 2024-05-08T23:39:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.635+0000 7f73affff700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 0x7f73b013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.640+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f73a4005580 con 0x7f73b006b660 2024-05-08T23:39:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.641+0000 7f73ad7fa700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f739406ad60 0x7f739406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.641+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f73a4088e80 con 0x7f73b006b660 2024-05-08T23:39:16.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.642+0000 7f73b51e3700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f739406ad60 0x7f739406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.643+0000 7f73b51e3700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f739406ad60 0x7f739406d220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009ef0 tx=0x7f73a8009480 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.644+0000 7f73affff700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73b0150f10 con 0x7f73b006b220 2024-05-08T23:39:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.662+0000 7f73affff700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 0x7f73b013e250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f739c0098f0 tx=0x7f739c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.664+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0045d0 con 0x7f73b006b220 2024-05-08T23:39:16.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.665+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f739c00b660 con 0x7f73b006b220 2024-05-08T23:39:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.665+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 msgr2=0x7f73b013e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.665+0000 7f73ad7fa700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 0x7f73b013e250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f739c0098f0 tx=0x7f739c004750 comp rx=0 tx=0).stop 2024-05-08T23:39:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.665+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 msgr2=0x7f73b013e250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:16.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.665+0000 7f73ad7fa700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b006b220 0x7f73b013e250 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f739c0098f0 tx=0x7f739c004750 comp rx=0 tx=0).stop 2024-05-08T23:39:16.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.690+0000 7f73b6c46700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 0x7f73b00ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:16.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.690+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0142520 con 0x7f73b0156fb0 2024-05-08T23:39:16.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.695+0000 7f73affff700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 0x7f73b00ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:16.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.697+0000 7f73affff700 1 -- 172.21.15.33:0/1738642255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f73b0070260 con 0x7f73b0156fb0 2024-05-08T23:39:16.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.698+0000 7f73affff700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 0x7f73b00ca2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f739c009c40 tx=0x7f739c0092c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:16.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.699+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c028990 con 0x7f73b0156fb0 2024-05-08T23:39:16.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.700+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4931 (secure 0 0 0) 0x7f739c032440 con 0x7f73b0156fb0 2024-05-08T23:39:16.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.700+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 msgr2=0x7f73b00ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.700+0000 7f73ad7fa700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 0x7f73b00ca2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f739c009c40 tx=0x7f739c0092c0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.700+0000 7f73ad7fa700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 msgr2=0x7f73b00ca2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:16.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.700+0000 7f73ad7fa700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b0156fb0 0x7f73b00ca2e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f739c009c40 tx=0x7f739c0092c0 comp rx=0 tx=0).stop 2024-05-08T23:39:16.698 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 56, 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:16.700 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:16.702 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:16.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:16.704 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:16.705 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:16.707 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:16.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:16.709 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:16.710 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:16.711 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.709+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f739406ad60 msgr2=0x7f739406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f739406ad60 0x7f739406d220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009ef0 tx=0x7f73a8009480 comp rx=0 tx=0).stop 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 msgr2=0x7f73b0144c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 --2- 172.21.15.33:0/1738642255 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b006b660 0x7f73b0144c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f73a400b550 tx=0x7f73a400b910 comp rx=0 tx=0).stop 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b51e3700 1 -- 172.21.15.33:0/1738642255 reap_dead start 2024-05-08T23:39:16.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b51e3700 1 -- 172.21.15.33:0/1738642255 reap_dead start 2024-05-08T23:39:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 shutdown_connections 2024-05-08T23:39:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 >> 172.21.15.33:0/1738642255 conn(0x7f73b00c8360 msgr2=0x7f73b006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 shutdown_connections 2024-05-08T23:39:16.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:16.710+0000 7f73b6c46700 1 -- 172.21.15.33:0/1738642255 wait complete. 2024-05-08T23:39:16.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:39:16.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:39:16.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:39:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.165+0000 7f0d8df50700 1 Processor -- start 2024-05-08T23:39:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.165+0000 7f0d8df50700 1 -- start start 2024-05-08T23:39:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0d880c7dd0 0x7f0d8806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 0x7f0d88158210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d88158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d88072ed0 con 0x7f0d880c3a90 2024-05-08T23:39:17.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d880c81b0 con 0x7f0d880c7dd0 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.167+0000 7f0d8df50700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880c2d60 con 0x7f0d880c8e50 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.170+0000 7f0d86ffd700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 msgr2=0x7f0d88158210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.170+0000 7f0d86ffd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 0x7f0d88158210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.170+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d88158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.171+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d88158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:43488/0 (socket says 172.21.15.33:43488) 2024-05-08T23:39:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.171+0000 7f0d87fff700 1 -- 172.21.15.33:0/130835220 learned_addr learned my addr 172.21.15.33:0/130835220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:17.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 --2- 172.21.15.33:0/130835220 >> v2:172.21.15.42:3300/0 conn(0x7f0d880c7dd0 0x7f0d8806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 -- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 msgr2=0x7f0d88158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:17.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 --2- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d88158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 -- 172.21.15.33:0/130835220 >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 msgr2=0x7f0d88158210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:17.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 --2- 172.21.15.33:0/130835220 >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 0x7f0d88158210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.174+0000 7f0d877fe700 1 -- 172.21.15.33:0/130835220 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88158d30 con 0x7f0d880c7dd0 2024-05-08T23:39:17.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.176+0000 7f0d877fe700 1 --2- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c7dd0 0x7f0d8806a990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d8806f060 tx=0x7f0d78009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1421eebf2f32bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:17.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.176+0000 7f0d867fc700 1 -- 172.21.15.33:0/130835220 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011830 con 0x7f0d880c7dd0 2024-05-08T23:39:17.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.176+0000 7f0d867fc700 1 -- 172.21.15.33:0/130835220 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78011990 con 0x7f0d880c7dd0 2024-05-08T23:39:17.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.176+0000 7f0d867fc700 1 -- 172.21.15.33:0/130835220 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011d20 con 0x7f0d880c7dd0 2024-05-08T23:39:17.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.177+0000 7f0d8df50700 1 -- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c7dd0 msgr2=0x7f0d8806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.177+0000 7f0d8df50700 1 --2- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c7dd0 0x7f0d8806a990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d8806f060 tx=0x7f0d78009b20 comp rx=0 tx=0).stop 2024-05-08T23:39:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.191+0000 7f0d8df50700 1 -- 172.21.15.33:0/130835220 shutdown_connections 2024-05-08T23:39:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.191+0000 7f0d8df50700 1 --2- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d88158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.191+0000 7f0d8df50700 1 --2- 172.21.15.33:0/130835220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c7dd0 0x7f0d8806a990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.191+0000 7f0d8df50700 1 --2- 172.21.15.33:0/130835220 >> v2:172.21.15.33:3300/0 conn(0x7f0d880c3a90 0x7f0d88158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.191+0000 7f0d8df50700 1 -- 172.21.15.33:0/130835220 >> 172.21.15.33:0/130835220 conn(0x7f0d880c8360 msgr2=0x7f0d880c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:17.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.193+0000 7f0d8df50700 1 -- 172.21.15.33:0/130835220 shutdown_connections 2024-05-08T23:39:17.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.194+0000 7f0d8df50700 1 -- 172.21.15.33:0/130835220 wait complete. 2024-05-08T23:39:17.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.196+0000 7f0d8df50700 1 Processor -- start 2024-05-08T23:39:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.197+0000 7f0d8df50700 1 -- start start 2024-05-08T23:39:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 0x7f0d880ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d880c7dd0 0x7f0d8814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d8814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88201000 con 0x7f0d880c7dd0 2024-05-08T23:39:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881ed6c0 con 0x7f0d880c3a90 2024-05-08T23:39:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d8df50700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88200130 con 0x7f0d880c8e50 2024-05-08T23:39:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.198+0000 7f0d86ffd700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d880c7dd0 msgr2=0x7f0d8814cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:39:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.199+0000 7f0d86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d880c7dd0 0x7f0d8814cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:17.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.200+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 0x7f0d880ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 0x7f0d880ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:42846/0 (socket says 172.21.15.33:42846) 2024-05-08T23:39:17.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 -- 172.21.15.33:0/62118383 learned_addr learned my addr 172.21.15.33:0/62118383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:17.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 msgr2=0x7f0d8814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d880c8e50 0x7f0d8814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d880c7dd0 msgr2=0x7f0d8814cd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:17.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d880c7dd0 0x7f0d8814cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.201+0000 7f0d877fe700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780094a0 con 0x7f0d880c3a90 2024-05-08T23:39:17.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.202+0000 7f0d877fe700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 0x7f0d880ca810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d780090b0 tx=0x7f0d78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.203+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780036a0 con 0x7f0d880c3a90 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.203+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7801f070 con 0x7f0d880c3a90 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.203+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003a80 con 0x7f0d880c3a90 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.203+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d881eb740 con 0x7f0d880c3a90 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.203+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88200660 con 0x7f0d880c3a90 2024-05-08T23:39:17.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.204+0000 7f0d8df50700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 0x7f0d88200820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.204+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88074a20 con 0x7f0d8814d8b0 2024-05-08T23:39:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.208+0000 7f0d86ffd700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 0x7f0d88200820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.209+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0d78003be0 con 0x7f0d880c3a90 2024-05-08T23:39:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.210+0000 7f0d6ffff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0d7406ac70 0x7f0d7406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.210+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0d78021070 con 0x7f0d880c3a90 2024-05-08T23:39:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.211+0000 7f0d86ffd700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d880c75a0 con 0x7f0d8814d8b0 2024-05-08T23:39:17.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.211+0000 7f0d86ffd700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 0x7f0d88200820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d88070a00 tx=0x7f0d7c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.212+0000 7f0d87fff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0d7406ac70 0x7f0d7406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.213+0000 7f0d87fff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0d7406ac70 0x7f0d7406d130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fa0 tx=0x7f0d8000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c003e50 con 0x7f0d8814d8b0 2024-05-08T23:39:17.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0d7c005070 con 0x7f0d8814d8b0 2024-05-08T23:39:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 msgr2=0x7f0d88200820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 0x7f0d88200820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d88070a00 tx=0x7f0d7c00b7e0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 msgr2=0x7f0d88200820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:17.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.222+0000 7f0d6ffff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d8814d8b0 0x7f0d88200820 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d88070a00 tx=0x7f0d7c00b7e0 comp rx=0 tx=0).stop 2024-05-08T23:39:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.253+0000 7f0d8df50700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 0x7f0d881fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.253+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880c3290 con 0x7f0d881fccc0 2024-05-08T23:39:17.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.255+0000 7f0d86ffd700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 0x7f0d881fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:17.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.256+0000 7f0d86ffd700 1 -- 172.21.15.33:0/62118383 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d880c2fb0 con 0x7f0d881fccc0 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.261+0000 7f0d86ffd700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 0x7f0d881fd0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c013000 tx=0x7f0d7c005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c01e070 con 0x7f0d881fccc0 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7f0d7c0058d0 con 0x7f0d881fccc0 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 msgr2=0x7f0d881fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 0x7f0d881fd0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c013000 tx=0x7f0d7c005a60 comp rx=0 tx=0).stop 2024-05-08T23:39:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 msgr2=0x7f0d881fd0d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.267+0000 7f0d6ffff700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0d881fccc0 0x7f0d881fd0d0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c013000 tx=0x7f0d7c005a60 comp rx=0 tx=0).stop 2024-05-08T23:39:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:17.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:17.272 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 56, 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:17.273 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:17.275 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:17.277 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.280 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:17.284 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:17.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:17.285 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:17.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:17.285 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:17.285 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0d7406ac70 msgr2=0x7f0d7406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d8df50700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0d7406ac70 0x7f0d7406d130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fa0 tx=0x7f0d8000b040 comp rx=0 tx=0).stop 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 msgr2=0x7f0d880ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d8df50700 1 --2- 172.21.15.33:0/62118383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d880c3a90 0x7f0d880ca810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d780090b0 tx=0x7f0d78004750 comp rx=0 tx=0).stop 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d87fff700 1 -- 172.21.15.33:0/62118383 reap_dead start 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.307+0000 7f0d87fff700 1 -- 172.21.15.33:0/62118383 reap_dead start 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.308+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 shutdown_connections 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.308+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 >> 172.21.15.33:0/62118383 conn(0x7f0d880c8360 msgr2=0x7f0d880c6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:17.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.308+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 shutdown_connections 2024-05-08T23:39:17.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:17.313+0000 7f0d8df50700 1 -- 172.21.15.33:0/62118383 wait complete. 2024-05-08T23:39:17.343 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:39:17.343 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:39:20.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:39:20.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:39:37.344 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:39:37.344 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-08T23:39:37.344 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:39:37.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:39:37.352 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:39:37.352 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:39:37.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:37.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.752+0000 7f4093981700 1 Processor -- start 2024-05-08T23:39:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.758+0000 7f4093981700 1 -- start start 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.759+0000 7f4093981700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 0x7f408c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.759+0000 7f4093981700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c15a2e0 con 0x7f408c0c67a0 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.759+0000 7f4093981700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408c15a580 con 0x7f408c0c9050 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.759+0000 7f4093981700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408c072720 con 0x7f408c1545c0 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.760+0000 7f4090f1c700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f408c0c9050 0x7f408c06a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:59378/0 (socket says 172.21.15.33:59378) 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.760+0000 7f4090f1c700 1 -- 172.21.15.33:0/3102889664 learned_addr learned my addr 172.21.15.33:0/3102889664 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.760+0000 7f409171d700 1 --2- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 0x7f408c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.761+0000 7f409171d700 1 -- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 msgr2=0x7f408c0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:37.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.761+0000 7f409171d700 1 -- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 msgr2=0x7f408c0c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.761+0000 7f409171d700 1 --2- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 0x7f408c0c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.761+0000 7f409171d700 1 --2- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 0x7f408c0c6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.767+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802380491 0 0) 0x7f408c15a580 con 0x7f408c0c9050 2024-05-08T23:39:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.767+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 --> v1:172.21.15.42:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40780035b0 con 0x7f408c0c9050 2024-05-08T23:39:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.1 v1:172.21.15.42:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3171477896 0 0) 0x7f40780035b0 con 0x7f408c0c9050 2024-05-08T23:39:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 msgr2=0x7f408c0c6260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 --2- 172.21.15.33:0/3102889664 >> v2:172.21.15.33:3300/0 conn(0x7f408c1545c0 0x7f408c0c6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 >> v1:172.21.15.33:6789/0 conn(0x7f408c0c67a0 legacy=0x7f408c06b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 --> v1:172.21.15.42:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c070ba0 con 0x7f408c0c9050 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.1 v1:172.21.15.42:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f40840019f0 con 0x7f408c0c9050 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 >> v1:172.21.15.42:6789/0 conn(0x7f408c0c9050 legacy=0x7f408c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4078003a50 0x7f4078003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078004850 0x7f4078004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 0x7f407800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c072720 con 0x7f4078005260 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407800c1a0 con 0x7f4078003a50 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.774+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407800c360 con 0x7f4078004850 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f409171d700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4078003a50 0x7f4078003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 0x7f407800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 -- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078004850 msgr2=0x7f4078004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078004850 0x7f4078004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 -- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4078003a50 msgr2=0x7f4078003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:37.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4078003a50 0x7f4078003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 -- 172.21.15.33:0/3102889664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c070ba0 con 0x7f4078005260 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.775+0000 7f4091f1e700 1 -- 172.21.15.33:0/3102889664 reap_dead start 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.776+0000 7f4091f1e700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 0x7f407800ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f407c001700 tx=0x7f4078005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a5986fe6e22bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.776+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0011c0 con 0x7f4078005260 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.776+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c0052f0 con 0x7f4078005260 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.776+0000 7f4083fff700 1 -- 172.21.15.33:0/3102889664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c003680 con 0x7f4078005260 2024-05-08T23:39:37.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.777+0000 7f4093981700 1 -- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 msgr2=0x7f407800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.777+0000 7f4093981700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 0x7f407800ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f407c001700 tx=0x7f4078005880 comp rx=0 tx=0).stop 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.778+0000 7f4093981700 1 -- 172.21.15.33:0/3102889664 shutdown_connections 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.778+0000 7f4093981700 1 --2- 172.21.15.33:0/3102889664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4078005260 0x7f407800ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.778+0000 7f4093981700 1 -- 172.21.15.33:0/3102889664 >> 172.21.15.33:0/3102889664 conn(0x7f408c0c8360 msgr2=0x7f408c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.778+0000 7f4093981700 1 -- 172.21.15.33:0/3102889664 shutdown_connections 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.778+0000 7f4093981700 1 -- 172.21.15.33:0/3102889664 wait complete. 2024-05-08T23:39:37.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.780+0000 7f4093981700 1 Processor -- start 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.781+0000 7f4093981700 1 -- start start 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.781+0000 7f4093981700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 0x7f408c152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.781+0000 7f4093981700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.781+0000 7f4093981700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f408c15bd40 0x7f408c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.781+0000 7f4093981700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c15d2f0 con 0x7f408c151520 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4093981700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c14ea10 con 0x7f408c15bd40 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4093981700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408c15b8b0 con 0x7f4078005260 2024-05-08T23:39:37.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49284/0 (socket says 172.21.15.33:49284) 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 -- 172.21.15.33:0/2604195957 learned_addr learned my addr 172.21.15.33:0/2604195957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 msgr2=0x7f408c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 msgr2=0x7f408c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.782+0000 7f4090f1c700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:37.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.783+0000 7f409171d700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 0x7f408c152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.789+0000 7f4091f1e700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f408c15bd40 0x7f408c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.790+0000 7f409171d700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f408c15bd40 msgr2=0x7f408c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.790+0000 7f409171d700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f408c15bd40 0x7f408c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.790+0000 7f409171d700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 msgr2=0x7f408c15de40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.790+0000 7f409171d700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:37.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.790+0000 7f409171d700 1 -- 172.21.15.33:0/2604195957 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c15d8f0 con 0x7f4078005260 2024-05-08T23:39:37.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.797+0000 7f409171d700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 0x7f408c152490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40880105e0 tx=0x7f40880109a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:37.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.800+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088015880 con 0x7f4078005260 2024-05-08T23:39:37.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.800+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408800c050 con 0x7f4078005260 2024-05-08T23:39:37.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.800+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088015c00 con 0x7f4078005260 2024-05-08T23:39:37.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.800+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c14d730 con 0x7f4078005260 2024-05-08T23:39:37.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.800+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c14dcf0 con 0x7f4078005260 2024-05-08T23:39:37.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.802+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c075460 con 0x7f4078005260 2024-05-08T23:39:37.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.813+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4088028480 con 0x7f4078005260 2024-05-08T23:39:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.814+0000 7f4081ffb700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 0x7f407006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.814+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f4088089cc0 con 0x7f4078005260 2024-05-08T23:39:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.814+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f408808a1b0 con 0x7f4078005260 2024-05-08T23:39:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.815+0000 7f4090f1c700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 0x7f407006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:37.825+0000 7f4090f1c700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 0x7f407006d1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4084001b20 tx=0x7f4084001cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:38.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.437+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f408c15d8f0 con 0x7f4078005260 2024-05-08T23:39:38.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.446+0000 7f4081ffb700 1 -- 172.21.15.33:0/2604195957 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f408c15d8f0 con 0x7f4078005260 2024-05-08T23:39:38.446 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":56,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 msgr2=0x7f407006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 0x7f407006d1d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4084001b20 tx=0x7f4084001cb0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 msgr2=0x7f408c152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 0x7f408c152490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40880105e0 tx=0x7f40880109a0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 shutdown_connections 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f407006ad10 0x7f407006d1d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f408c15bd40 0x7f408c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4078005260 0x7f408c152490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 --2- 172.21.15.33:0/2604195957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c151520 0x7f408c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.454+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 >> 172.21.15.33:0/2604195957 conn(0x7f408c0c8360 msgr2=0x7f408c0c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.455+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 shutdown_connections 2024-05-08T23:39:38.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:38.455+0000 7f4093981700 1 -- 172.21.15.33:0/2604195957 wait complete. 2024-05-08T23:39:41.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.968+0000 7fab6a0f5700 1 Processor -- start 2024-05-08T23:39:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.968+0000 7fab6a0f5700 1 -- start start 2024-05-08T23:39:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.969+0000 7fab6a0f5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.969+0000 7fab6a0f5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.969+0000 7fab6a0f5700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6406ba40 con 0x7fab640c2770 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.969+0000 7fab6a0f5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6415f180 con 0x7fab640c7d80 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.969+0000 7fab6a0f5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab640c8210 con 0x7fab641589d0 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.975+0000 7fab637fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.975+0000 7fab637fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:58402/0 (socket says 172.21.15.33:58402) 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.975+0000 7fab637fe700 1 -- 172.21.15.33:0/3076366881 learned_addr learned my addr 172.21.15.33:0/3076366881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.977+0000 7fab63fff700 1 --2- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.979+0000 7fab63fff700 1 -- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 msgr2=0x7fab64070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.979+0000 7fab63fff700 1 -- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 msgr2=0x7fab64070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.979+0000 7fab63fff700 1 --2- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.979+0000 7fab63fff700 1 --2- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.981+0000 7fab637fe700 1 -- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 msgr2=0x7fab64070dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.981+0000 7fab637fe700 1 --2- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.981+0000 7fab637fe700 1 -- 172.21.15.33:0/3076366881 >> v1:172.21.15.33:6789/0 conn(0x7fab640c2770 legacy=0x7fab6406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.981+0000 7fab637fe700 1 -- 172.21.15.33:0/3076366881 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64070930 con 0x7fab641589d0 2024-05-08T23:39:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.983+0000 7fab637fe700 1 --2- 172.21.15.33:0/3076366881 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab540098f0 tx=0x7fab54009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb56389465e3b97d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.992+0000 7fab627fc700 1 -- 172.21.15.33:0/3076366881 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400b8a0 con 0x7fab641589d0 2024-05-08T23:39:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.992+0000 7fab627fc700 1 -- 172.21.15.33:0/3076366881 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5400ba00 con 0x7fab641589d0 2024-05-08T23:39:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.992+0000 7fab627fc700 1 -- 172.21.15.33:0/3076366881 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400bd70 con 0x7fab641589d0 2024-05-08T23:39:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.993+0000 7fab6a0f5700 1 -- 172.21.15.33:0/3076366881 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 msgr2=0x7fab640c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.993+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/3076366881 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab540098f0 tx=0x7fab54009c00 comp rx=0 tx=0).stop 2024-05-08T23:39:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.994+0000 7fab6a0f5700 1 -- 172.21.15.33:0/3076366881 shutdown_connections 2024-05-08T23:39:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.994+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/3076366881 >> v2:172.21.15.33:3300/0 conn(0x7fab640c7d80 0x7fab64070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.994+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/3076366881 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab640c21a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:41.994+0000 7fab6a0f5700 1 -- 172.21.15.33:0/3076366881 >> 172.21.15.33:0/3076366881 conn(0x7fab640c8360 msgr2=0x7fab640c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:42.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.000+0000 7fab6a0f5700 1 -- 172.21.15.33:0/3076366881 shutdown_connections 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.001+0000 7fab6a0f5700 1 -- 172.21.15.33:0/3076366881 wait complete. 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.003+0000 7fab6a0f5700 1 Processor -- start 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.003+0000 7fab6a0f5700 1 -- start start 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 0x7fab641302d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab64134a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab64131b70 con 0x7fab640c7d80 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab64130d60 con 0x7fab640c2770 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.004+0000 7fab6a0f5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab641581a0 con 0x7fab641589d0 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.005+0000 7fab637fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 0x7fab641302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49306/0 (socket says 172.21.15.33:49306) 2024-05-08T23:39:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 -- 172.21.15.33:0/1663231488 learned_addr learned my addr 172.21.15.33:0/1663231488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 msgr2=0x7fab64157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 msgr2=0x7fab64157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.006+0000 7fab62ffd700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.007+0000 7fab637fe700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 msgr2=0x7fab64134a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.007+0000 7fab637fe700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab64134a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.007+0000 7fab637fe700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 msgr2=0x7fab64157c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.007+0000 7fab637fe700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.007+0000 7fab637fe700 1 -- 172.21.15.33:0/1663231488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009550 con 0x7fab640c2770 2024-05-08T23:39:42.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.020+0000 7fab637fe700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 0x7fab641302d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab54016000 tx=0x7fab54004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.031+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54004590 con 0x7fab640c2770 2024-05-08T23:39:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.031+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5401f070 con 0x7fab640c2770 2024-05-08T23:39:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.031+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab540038c0 con 0x7fab640c2770 2024-05-08T23:39:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.031+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab640c4c00 con 0x7fab640c2770 2024-05-08T23:39:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.031+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab64131260 con 0x7fab640c2770 2024-05-08T23:39:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.035+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab64075460 con 0x7fab640c2770 2024-05-08T23:39:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.036+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fab54003a20 con 0x7fab640c2770 2024-05-08T23:39:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.036+0000 7fab4bfff700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 0x7fab5006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.036+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fab54021070 con 0x7fab640c2770 2024-05-08T23:39:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.037+0000 7fab62ffd700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 0x7fab5006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.038+0000 7fab62ffd700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 0x7fab5006d220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab64140b60 tx=0x7fab58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.046+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fab54055c90 con 0x7fab640c2770 2024-05-08T23:39:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.728+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fab640ca330 con 0x7fab640c2770 2024-05-08T23:39:42.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.734+0000 7fab4bfff700 1 -- 172.21.15.33:0/1663231488 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fab54015610 con 0x7fab640c2770 2024-05-08T23:39:42.737 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":56,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.755+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 msgr2=0x7fab5006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.755+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 0x7fab5006d220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab64140b60 tx=0x7fab58009040 comp rx=0 tx=0).stop 2024-05-08T23:39:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.755+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 msgr2=0x7fab641302d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.755+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 0x7fab641302d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab54016000 tx=0x7fab54004750 comp rx=0 tx=0).stop 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 shutdown_connections 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fab5006ad60 0x7fab5006d220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fab641589d0 0x7fab64134a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab640c2770 0x7fab641302d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 --2- 172.21.15.33:0/1663231488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab640c7d80 0x7fab64157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.757+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 >> 172.21.15.33:0/1663231488 conn(0x7fab640c8360 msgr2=0x7fab640c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.758+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 shutdown_connections 2024-05-08T23:39:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:42.758+0000 7fab6a0f5700 1 -- 172.21.15.33:0/1663231488 wait complete. 2024-05-08T23:39:45.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.107+0000 7f8140797700 1 Processor -- start 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.107+0000 7f8140797700 1 -- start start 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.108+0000 7f8140797700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.108+0000 7f8140797700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813806ba40 con 0x7f81380c7d80 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.108+0000 7f8140797700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8138070930 con 0x7f81381589d0 2024-05-08T23:39:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.108+0000 7f8140797700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813815f180 con 0x7f81380c2770 2024-05-08T23:39:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.111+0000 7f813ed34700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 0x7f8138070dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33598/0 (socket says 172.21.15.33:33598) 2024-05-08T23:39:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.111+0000 7f813ed34700 1 -- 172.21.15.33:0/1533654915 learned_addr learned my addr 172.21.15.33:0/1533654915 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.111+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.112+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.112+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.112+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.112+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:46.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.318+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.318+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.318+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.318+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:46.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.318+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.721+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.721+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.721+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.721+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:46.721+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.525+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.525+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T23:39:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.525+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.525+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.525+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:47.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.751+0000 7f813ed34700 1 -- 172.21.15.33:0/1533654915 >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 legacy=0x7f8138070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T23:39:47.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.752+0000 7f813ed34700 1 -- 172.21.15.33:0/1533654915 >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 legacy=0x7f8138070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:39:47.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.752+0000 7f813ed34700 1 --1- 172.21.15.33:0/1533654915 >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 0x7f8138070dc0 :-1 s=OPENED pgs=27 cs=1 l=1).handle_message read tag failed 2024-05-08T23:39:47.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:47.752+0000 7f813ed34700 1 --1- 172.21.15.33:0/1533654915 >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 0x7f8138070dc0 :-1 s=OPENED pgs=27 cs=1 l=1).fault on lossy channel, failing 2024-05-08T23:39:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.110+0000 7f812ffff700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 msgr2=0x7f81380c21a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:49.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.110+0000 7f812ffff700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.42:3300/0 conn(0x7f81380c2770 0x7f81380c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 -- 172.21.15.33:0/1533654915 >> v1:172.21.15.42:6789/0 conn(0x7f81381589d0 legacy=0x7f813806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 -- 172.21.15.33:0/1533654915 >> v1:172.21.15.33:6789/0 conn(0x7f81380c7d80 legacy=0x7f8138070dc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 -- 172.21.15.33:0/1533654915 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c008cf0 con 0x7f811c003680 2024-05-08T23:39:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.111+0000 7f812ffff700 1 -- 172.21.15.33:0/1533654915 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c008e60 con 0x7f811c006140 2024-05-08T23:39:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.112+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.113+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.113+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.113+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.113+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.119+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.120+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:39:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.120+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.120+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.120+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:39:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.316+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.316+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.316+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.316+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.317+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.324+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.324+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.324+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.324+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.324+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:39:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.720+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.720+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.720+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.720+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.720+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.725+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.725+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.725+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.725+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:49.725+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:39:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.521+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.521+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:39:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.521+0000 7f813e533700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:39:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.521+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:39:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.521+0000 7f813e533700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:39:50.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.529+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.529+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 msgr2=0x7f811c005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:50.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.529+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> v2:172.21.15.33:3300/0 conn(0x7f811c003680 0x7f811c005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:50.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.529+0000 7f813dd32700 1 -- 172.21.15.33:0/1533654915 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813806b4c0 con 0x7f811c006140 2024-05-08T23:39:50.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.530+0000 7f813dd32700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8128009100 tx=0x7f8128018cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510f08fe1ff3ed2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.542+0000 7f813d531700 1 -- 172.21.15.33:0/1533654915 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812801d070 con 0x7f811c006140 2024-05-08T23:39:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.542+0000 7f813d531700 1 -- 172.21.15.33:0/1533654915 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800c050 con 0x7f811c006140 2024-05-08T23:39:50.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.542+0000 7f813d531700 1 -- 172.21.15.33:0/1533654915 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128018250 con 0x7f811c006140 2024-05-08T23:39:50.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.544+0000 7f8140797700 1 -- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f811c008600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.544+0000 7f8140797700 1 --2- 172.21.15.33:0/1533654915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f811c008600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8128009100 tx=0x7f8128018cc0 comp rx=0 tx=0).stop 2024-05-08T23:39:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.544+0000 7f813ed34700 1 -- 172.21.15.33:0/1533654915 reap_dead start 2024-05-08T23:39:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.544+0000 7f8140797700 1 -- 172.21.15.33:0/1533654915 shutdown_connections 2024-05-08T23:39:50.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.544+0000 7f8140797700 1 -- 172.21.15.33:0/1533654915 >> 172.21.15.33:0/1533654915 conn(0x7f81380c8360 msgr2=0x7f81380c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:50.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.548+0000 7f8140797700 1 -- 172.21.15.33:0/1533654915 shutdown_connections 2024-05-08T23:39:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.548+0000 7f8140797700 1 -- 172.21.15.33:0/1533654915 wait complete. 2024-05-08T23:39:50.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.550+0000 7f8140797700 1 Processor -- start 2024-05-08T23:39:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.551+0000 7f8140797700 1 -- start start 2024-05-08T23:39:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f81381fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f813812ce00 0x7f8138154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 0x7f813815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138200330 con 0x7f8138200750 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81381555d0 con 0x7f813812ce00 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.552+0000 7f8140797700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813815e260 con 0x7f811c006140 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.553+0000 7f813e533700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f81381fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.553+0000 7f813e533700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f81381fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:46182/0 (socket says 172.21.15.33:46182) 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.553+0000 7f813e533700 1 -- 172.21.15.33:0/1922715219 learned_addr learned my addr 172.21.15.33:0/1922715219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.553+0000 7f813ed34700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 0x7f813815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813dd32700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f813812ce00 0x7f8138154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 -- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 msgr2=0x7f81381fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f81381fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 -- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f813812ce00 msgr2=0x7f8138154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f813812ce00 0x7f8138154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 -- 172.21.15.33:0/1922715219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812800e040 con 0x7f8138200750 2024-05-08T23:39:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.554+0000 7f813ed34700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 0x7f813815c5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f813806f070 tx=0x7f813400c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.555+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134005b40 con 0x7f8138200750 2024-05-08T23:39:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.555+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134005ca0 con 0x7f8138200750 2024-05-08T23:39:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.555+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134007290 con 0x7f8138200750 2024-05-08T23:39:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.556+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813815dca0 con 0x7f8138200750 2024-05-08T23:39:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.556+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81381ffc70 con 0x7f8138200750 2024-05-08T23:39:50.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.558+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8138200470 con 0x7f8138200750 2024-05-08T23:39:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.577+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f81340073f0 con 0x7f8138200750 2024-05-08T23:39:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.577+0000 7f812effd700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 0x7f811c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.578+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8134088540 con 0x7f8138200750 2024-05-08T23:39:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.578+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f81340889c0 con 0x7f8138200750 2024-05-08T23:39:50.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.578+0000 7f813e533700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 0x7f811c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:50.580+0000 7f813e533700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 0x7f811c06e100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8130001730 tx=0x7f8130009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.096+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8138200470 con 0x7f8138200750 2024-05-08T23:39:51.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.096+0000 7f812effd700 1 -- 172.21.15.33:0/1922715219 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f8138200470 con 0x7f8138200750 2024-05-08T23:39:51.094 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 msgr2=0x7f811c06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 0x7f811c06e100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8130001730 tx=0x7f8130009040 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 msgr2=0x7f813815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 0x7f813815c5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f813806f070 tx=0x7f813400c670 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 shutdown_connections 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811c06bc40 0x7f811c06e100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8138200750 0x7f813815c5c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f811c006140 0x7f81381fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 --2- 172.21.15.33:0/1922715219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f813812ce00 0x7f8138154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 >> 172.21.15.33:0/1922715219 conn(0x7f81380c8360 msgr2=0x7f81380c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 shutdown_connections 2024-05-08T23:39:51.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.099+0000 7f8140797700 1 -- 172.21.15.33:0/1922715219 wait complete. 2024-05-08T23:39:51.113 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:39:51.113 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:39:51.113 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:39:51.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:39:51.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:39:51.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.272+0000 7f8446818700 1 Processor -- start 2024-05-08T23:39:51.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.272+0000 7f8446818700 1 -- start start 2024-05-08T23:39:51.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.273+0000 7f8446818700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 0x7f84401567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.273+0000 7f8446818700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.273+0000 7f8446818700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8440150f90 con 0x7f84400c8080 2024-05-08T23:39:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.273+0000 7f8446818700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844015a690 con 0x7f84400c61b0 2024-05-08T23:39:51.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.273+0000 7f8446818700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8440159d30 con 0x7f8440150bb0 2024-05-08T23:39:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.275+0000 7f843f7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.275+0000 7f843f7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:45606/0 (socket says 172.21.15.33:45606) 2024-05-08T23:39:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.275+0000 7f843f7fe700 1 -- 172.21.15.33:0/751858136 learned_addr learned my addr 172.21.15.33:0/751858136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.275+0000 7f8444db5700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f84400c8080 0x7f84400c6dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:56792/0 (socket says 172.21.15.33:56792) 2024-05-08T23:39:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.275+0000 7f843ffff700 1 --2- 172.21.15.33:0/751858136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 0x7f84401567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.276+0000 7f843ffff700 1 -- 172.21.15.33:0/751858136 >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 msgr2=0x7f84400c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.276+0000 7f843ffff700 1 --2- 172.21.15.33:0/751858136 >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.276+0000 7f843ffff700 1 -- 172.21.15.33:0/751858136 >> v1:172.21.15.42:6789/0 conn(0x7f84400c8080 legacy=0x7f84400c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.276+0000 7f843ffff700 1 -- 172.21.15.33:0/751858136 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400c85f0 con 0x7f8440150bb0 2024-05-08T23:39:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.276+0000 7f843f7fe700 1 --2- 172.21.15.33:0/751858136 >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:39:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.277+0000 7f843ffff700 1 --2- 172.21.15.33:0/751858136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 0x7f84401567c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f8430009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de864ec90d4e107 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.277+0000 7f843effd700 1 -- 172.21.15.33:0/751858136 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430011830 con 0x7f8440150bb0 2024-05-08T23:39:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.277+0000 7f843effd700 1 -- 172.21.15.33:0/751858136 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430011990 con 0x7f8440150bb0 2024-05-08T23:39:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.277+0000 7f843effd700 1 -- 172.21.15.33:0/751858136 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430011d20 con 0x7f8440150bb0 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 -- 172.21.15.33:0/751858136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 msgr2=0x7f84401567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 --2- 172.21.15.33:0/751858136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 0x7f84401567c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f8430009b50 comp rx=0 tx=0).stop 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 -- 172.21.15.33:0/751858136 shutdown_connections 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 --2- 172.21.15.33:0/751858136 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8440150bb0 0x7f84401567c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 --2- 172.21.15.33:0/751858136 >> v2:172.21.15.42:3300/0 conn(0x7f84400c61b0 0x7f84400c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 -- 172.21.15.33:0/751858136 >> 172.21.15.33:0/751858136 conn(0x7f844006a990 msgr2=0x7f844006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 -- 172.21.15.33:0/751858136 shutdown_connections 2024-05-08T23:39:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.278+0000 7f8446818700 1 -- 172.21.15.33:0/751858136 wait complete. 2024-05-08T23:39:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.280+0000 7f8446818700 1 Processor -- start 2024-05-08T23:39:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.280+0000 7f8446818700 1 -- start start 2024-05-08T23:39:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400c61b0 0x7f84400c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84400c8080 0x7f8440155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 0x7f84401561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440154ea0 con 0x7f8440150bb0 2024-05-08T23:39:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401f3d30 con 0x7f84400c61b0 2024-05-08T23:39:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.281+0000 7f8446818700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84401f3fd0 con 0x7f84400c8080 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84400c8080 0x7f8440155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 0x7f84401561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84400c8080 0x7f8440155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:46192/0 (socket says 172.21.15.33:46192) 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f843f7fe700 1 -- 172.21.15.33:0/3261517862 learned_addr learned my addr 172.21.15.33:0/3261517862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 0x7f84401561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46056/0 (socket says 172.21.15.33:46056) 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f843ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400c61b0 0x7f84400c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84400c8080 msgr2=0x7f8440155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f84400c8080 0x7f8440155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400c61b0 msgr2=0x7f84400c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400c61b0 0x7f84400c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.282+0000 7f8444db5700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843400a6f0 con 0x7f8440150bb0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f8444db5700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 0x7f84401561d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f843800ca70 tx=0x7f843800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843800a9e0 con 0x7f8440150bb0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843800ab40 con 0x7f8440150bb0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438007070 con 0x7f8440150bb0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84300094a0 con 0x7f8440150bb0 2024-05-08T23:39:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.283+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84401e1350 con 0x7f8440150bb0 2024-05-08T23:39:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.284+0000 7f8446818700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 0x7f8440145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.284+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440074900 con 0x7f8440157380 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.284+0000 7f843ffff700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 0x7f8440145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.284+0000 7f843ffff700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f844015a2b0 con 0x7f8440157380 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843ffff700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 0x7f8440145a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f84300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430020040 con 0x7f8440157380 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f84300045e0 con 0x7f8440157380 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 msgr2=0x7f8440145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 0x7f8440145a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f84300047c0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 msgr2=0x7f8440145a90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440157380 0x7f8440145a90 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8430009840 tx=0x7f84300047c0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.285+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f84380071d0 con 0x7f8440150bb0 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.286+0000 7f843cff9700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f842c06a960 0x7f842c06ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.286+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8438088d30 con 0x7f8440150bb0 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.286+0000 7f843ffff700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f842c06a960 0x7f842c06ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.287+0000 7f843ffff700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f842c06a960 0x7f842c06ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8430025000 tx=0x7f843001e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.297+0000 7f8446818700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 0x7f844014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.297+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440159fd0 con 0x7f8440145fd0 2024-05-08T23:39:51.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.297+0000 7f843f7fe700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 0x7f844014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843f7fe700 1 -- 172.21.15.33:0/3261517862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f844015aab0 con 0x7f8440145fd0 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843f7fe700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 0x7f844014f430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84401f42b0 tx=0x7f8434011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400fc90 con 0x7f8440145fd0 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4459 (secure 0 0 0) 0x7f843400fdf0 con 0x7f8440145fd0 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 msgr2=0x7f844014f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 0x7f844014f430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84401f42b0 tx=0x7f8434011db0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 msgr2=0x7f844014f430 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.298+0000 7f843cff9700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440145fd0 0x7f844014f430 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84401f42b0 tx=0x7f8434011db0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:51.309 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f842c06a960 msgr2=0x7f842c06ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f842c06a960 0x7f842c06ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8430025000 tx=0x7f843001e000 comp rx=0 tx=0).stop 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 msgr2=0x7f84401561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 --2- 172.21.15.33:0/3261517862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8440150bb0 0x7f84401561d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f843800ca70 tx=0x7f843800cd80 comp rx=0 tx=0).stop 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8444db5700 1 -- 172.21.15.33:0/3261517862 reap_dead start 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8444db5700 1 -- 172.21.15.33:0/3261517862 reap_dead start 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 shutdown_connections 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 >> 172.21.15.33:0/3261517862 conn(0x7f844006a990 msgr2=0x7f84400667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 shutdown_connections 2024-05-08T23:39:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.299+0000 7f8446818700 1 -- 172.21.15.33:0/3261517862 wait complete. 2024-05-08T23:39:51.313 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:39:51.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:39:51.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:39:51.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.520+0000 7f8356e47700 1 Processor -- start 2024-05-08T23:39:51.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.520+0000 7f8356e47700 1 -- start start 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8356e47700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f83500c8470 0x7f83500c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8356e47700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f83500c7490 0x7f835015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8356e47700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835015db10 con 0x7f83501543b0 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8356e47700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835015a3f0 con 0x7f83500c7490 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8356e47700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835015ddb0 con 0x7f83500c8470 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8354be3700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f83501543b0 0x7f8350066770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33602/0 (socket says 172.21.15.33:33602) 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f8354be3700 1 -- 172.21.15.33:0/3424059967 learned_addr learned my addr 172.21.15.33:0/3424059967 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:51.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f834ffff700 1 --2- 172.21.15.33:0/3424059967 >> v2:172.21.15.42:3300/0 conn(0x7f83500c8470 0x7f83500c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f83553e4700 1 --2- 172.21.15.33:0/3424059967 >> v2:172.21.15.33:3300/0 conn(0x7f83500c7490 0x7f835015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f83553e4700 1 -- 172.21.15.33:0/3424059967 >> v2:172.21.15.42:3300/0 conn(0x7f83500c8470 msgr2=0x7f83500c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f83553e4700 1 --2- 172.21.15.33:0/3424059967 >> v2:172.21.15.42:3300/0 conn(0x7f83500c8470 0x7f83500c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f83553e4700 1 -- 172.21.15.33:0/3424059967 >> v1:172.21.15.33:6789/0 conn(0x7f83501543b0 legacy=0x7f8350066770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.521+0000 7f83553e4700 1 -- 172.21.15.33:0/3424059967 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835015e090 con 0x7f83500c7490 2024-05-08T23:39:51.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.522+0000 7f83553e4700 1 --2- 172.21.15.33:0/3424059967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c7490 0x7f835015a810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8350072d40 tx=0x7f834800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24500ff877516fe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.524+0000 7f834f7fe700 1 -- 172.21.15.33:0/3424059967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800e070 con 0x7f83500c7490 2024-05-08T23:39:51.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.524+0000 7f834f7fe700 1 -- 172.21.15.33:0/3424059967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800bb20 con 0x7f83500c7490 2024-05-08T23:39:51.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.524+0000 7f834f7fe700 1 -- 172.21.15.33:0/3424059967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348012430 con 0x7f83500c7490 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 -- 172.21.15.33:0/3424059967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c7490 msgr2=0x7f835015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 --2- 172.21.15.33:0/3424059967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c7490 0x7f835015a810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8350072d40 tx=0x7f834800b330 comp rx=0 tx=0).stop 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 -- 172.21.15.33:0/3424059967 shutdown_connections 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 --2- 172.21.15.33:0/3424059967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c7490 0x7f835015a810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 --2- 172.21.15.33:0/3424059967 >> v2:172.21.15.42:3300/0 conn(0x7f83500c8470 0x7f83500c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 -- 172.21.15.33:0/3424059967 >> 172.21.15.33:0/3424059967 conn(0x7f835006a990 msgr2=0x7f835006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 -- 172.21.15.33:0/3424059967 shutdown_connections 2024-05-08T23:39:51.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.525+0000 7f8356e47700 1 -- 172.21.15.33:0/3424059967 wait complete. 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.527+0000 7f8356e47700 1 Processor -- start 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.527+0000 7f8356e47700 1 -- start start 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83500c7490 0x7f8350145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501543b0 0x7f835015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835015b3a0 con 0x7f83500c8470 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83501474c0 con 0x7f83501543b0 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8356e47700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835015cc20 con 0x7f83500c7490 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46090/0 (socket says 172.21.15.33:46090) 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 -- 172.21.15.33:0/624730027 learned_addr learned my addr 172.21.15.33:0/624730027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f8354be3700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83500c7490 0x7f8350145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83500c7490 msgr2=0x7f8350145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f83500c7490 0x7f8350145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501543b0 msgr2=0x7f835015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501543b0 0x7f835015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009a70 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.528+0000 7f834ffff700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e9b0 tx=0x7f833c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.529+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c009710 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.529+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c00c850 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.529+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83501458d0 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.529+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00cbc0 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.529+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835015b580 con 0x7f83500c8470 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.530+0000 7f8356e47700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 0x7f83501f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.530+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350074900 con 0x7f835015b7a0 2024-05-08T23:39:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.530+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f833c00cd20 con 0x7f83500c8470 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.531+0000 7f834d7fa700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833406ac90 0x7f833406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.531+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f833c089590 con 0x7f83500c8470 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.531+0000 7f83553e4700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833406ac90 0x7f833406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.531+0000 7f83553e4700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833406ac90 0x7f833406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f835014f230 tx=0x7f8348010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.532+0000 7f8354be3700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 0x7f83501f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.533+0000 7f8354be3700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f835015e650 con 0x7f835015b7a0 2024-05-08T23:39:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.533+0000 7f8354be3700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 0x7f83501f2d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f834400e860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.536+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344005b50 con 0x7f835015b7a0 2024-05-08T23:39:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.537+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8344005cb0 con 0x7f835015b7a0 2024-05-08T23:39:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.537+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 msgr2=0x7f83501f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.537+0000 7f834d7fa700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 0x7f83501f2d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f834400e860 comp rx=0 tx=0).stop 2024-05-08T23:39:51.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.537+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 msgr2=0x7f83501f2d20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.537+0000 7f834d7fa700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f835015b7a0 0x7f83501f2d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.545+0000 7f8356e47700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 0x7f83501f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.545+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835015d7d0 con 0x7f83501f3260 2024-05-08T23:39:51.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.545+0000 7f8354be3700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 0x7f83501f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.545+0000 7f8354be3700 1 -- 172.21.15.33:0/624730027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8350154790 con 0x7f83501f3260 2024-05-08T23:39:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.545+0000 7f8354be3700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 0x7f83501f3670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f8344005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344021070 con 0x7f83501f3260 2024-05-08T23:39:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4940 (secure 0 0 0) 0x7f834401c8e0 con 0x7f83501f3260 2024-05-08T23:39:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 msgr2=0x7f83501f3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 0x7f83501f3670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f8344005880 comp rx=0 tx=0).stop 2024-05-08T23:39:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 msgr2=0x7f83501f3670 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.549+0000 7f834d7fa700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83501f3260 0x7f83501f3670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.561 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:51.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.562 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:51.562 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833406ac90 msgr2=0x7f833406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f833406ac90 0x7f833406d150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f835014f230 tx=0x7f8348010000 comp rx=0 tx=0).stop 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 msgr2=0x7f83501491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e9b0 tx=0x7f833c00ed70 comp rx=0 tx=0).stop 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f83553e4700 1 -- 172.21.15.33:0/624730027 reap_dead start 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 shutdown_connections 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 --2- 172.21.15.33:0/624730027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83500c8470 0x7f83501491c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 >> 172.21.15.33:0/624730027 conn(0x7f835006a990 msgr2=0x7f83500713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 shutdown_connections 2024-05-08T23:39:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.550+0000 7f8356e47700 1 -- 172.21.15.33:0/624730027 wait complete. 2024-05-08T23:39:51.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:39:51.564 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:39:51.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:39:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.726+0000 7f7682162700 1 Processor -- start 2024-05-08T23:39:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.726+0000 7f7682162700 1 -- start start 2024-05-08T23:39:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f7682162700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f767c154440 0x7f767c070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f7682162700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c15e750 con 0x7f767c0c6e30 2024-05-08T23:39:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f7682162700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0709a0 con 0x7f767c0c7210 2024-05-08T23:39:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f7682162700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c15dbd0 con 0x7f767c154440 2024-05-08T23:39:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f767b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f767c0c6e30 0x7f767c15ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33610/0 (socket says 172.21.15.33:33610) 2024-05-08T23:39:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.727+0000 7f767b7fe700 1 -- 172.21.15.33:0/1907807186 learned_addr learned my addr 172.21.15.33:0/1907807186 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:39:51.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562753442 0 0) 0x7f767c15e750 con 0x7f767c0c6e30 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76640035b0 con 0x7f767c0c6e30 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885163994 0 0) 0x7f76640035b0 con 0x7f767c0c6e30 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 >> v2:172.21.15.42:3300/0 conn(0x7f767c154440 msgr2=0x7f767c070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 --2- 172.21.15.33:0/1907807186 >> v2:172.21.15.42:3300/0 conn(0x7f767c154440 0x7f767c070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 >> v1:172.21.15.42:6789/0 conn(0x7f767c0c7210 legacy=0x7f767c153d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c15deb0 con 0x7f767c0c6e30 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f766c001670 con 0x7f767c0c6e30 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 >> v1:172.21.15.33:6789/0 conn(0x7f767c0c6e30 legacy=0x7f767c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664003a00 0x7f7664003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664004850 0x7f7664004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 0x7f766400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c15dbd0 con 0x7f7664005330 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766400c230 con 0x7f7664003a00 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766400c3f0 con 0x7f7664004850 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767b7fe700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664003a00 0x7f7664003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.728+0000 7f767bfff700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 0x7f766400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 -- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664004850 msgr2=0x7f7664004cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664004850 0x7f7664004cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 -- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664003a00 msgr2=0x7f7664003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664003a00 0x7f7664003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 -- 172.21.15.33:0/1907807186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640035b0 con 0x7f7664005330 2024-05-08T23:39:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 -- 172.21.15.33:0/1907807186 reap_dead start 2024-05-08T23:39:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767bfff700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 0x7f766400ba10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f767c15a200 tx=0x7f7664005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2189dddf3ed4bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76740098b0 con 0x7f7664005330 2024-05-08T23:39:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674003c20 con 0x7f7664005330 2024-05-08T23:39:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.729+0000 7f767a7fc700 1 -- 172.21.15.33:0/1907807186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400b870 con 0x7f7664005330 2024-05-08T23:39:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.730+0000 7f7682162700 1 -- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 msgr2=0x7f766400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.730+0000 7f7682162700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 0x7f766400ba10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f767c15a200 tx=0x7f7664005880 comp rx=0 tx=0).stop 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.730+0000 7f7682162700 1 -- 172.21.15.33:0/1907807186 shutdown_connections 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.730+0000 7f7682162700 1 --2- 172.21.15.33:0/1907807186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7664005330 0x7f766400ba10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.730+0000 7f7682162700 1 -- 172.21.15.33:0/1907807186 >> 172.21.15.33:0/1907807186 conn(0x7f767c06a990 msgr2=0x7f767c06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.731+0000 7f7682162700 1 -- 172.21.15.33:0/1907807186 shutdown_connections 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.731+0000 7f7682162700 1 -- 172.21.15.33:0/1907807186 wait complete. 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.732+0000 7f7682162700 1 Processor -- start 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 -- start start 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664005330 0x7f767c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f767c15b8a0 0x7f767c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 0x7f767c149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c14f300 con 0x7f767c0c56e0 2024-05-08T23:39:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c14cbd0 con 0x7f767c15b8a0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.733+0000 7f7682162700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c14f550 con 0x7f7664005330 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 0x7f767c149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 0x7f767c149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46098/0 (socket says 172.21.15.33:46098) 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 learned_addr learned my addr 172.21.15.33:0/4048125175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664005330 msgr2=0x7f767c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f7664005330 0x7f767c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f767c15b8a0 msgr2=0x7f767c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f767c15b8a0 0x7f767c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c14f830 con 0x7f767c0c56e0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f767bfff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 0x7f767c149ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f767c070040 tx=0x7f7674004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76740092b0 con 0x7f767c0c56e0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674015b50 con 0x7f767c0c56e0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c1efe40 con 0x7f767c0c56e0 2024-05-08T23:39:51.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400d410 con 0x7f767c0c56e0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.734+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c1f0400 con 0x7f767c0c56e0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.735+0000 7f7682162700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 0x7f767c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.735+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c074900 con 0x7f767c15d6c0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.736+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f767400d570 con 0x7f767c0c56e0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.736+0000 7f7663fff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f766806ac90 0x7f766806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.737+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f7674011070 con 0x7f767c0c56e0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.737+0000 7f767affd700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f766806ac90 0x7f766806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.737+0000 7f767affd700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f766806ac90 0x7f766806d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f767c04f3a0 tx=0x7f7670006cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.739+0000 7f767b7fe700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 0x7f767c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.740+0000 7f767b7fe700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f767c15e470 con 0x7f767c15d6c0 2024-05-08T23:39:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.740+0000 7f767b7fe700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 0x7f767c14a010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f767c0647e0 tx=0x7f766c007620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c007800 con 0x7f767c15d6c0 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f766c00f720 con 0x7f767c15d6c0 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 msgr2=0x7f767c14a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 0x7f767c14a010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f767c0647e0 tx=0x7f766c007620 comp rx=0 tx=0).stop 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 msgr2=0x7f767c14a010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.742+0000 7f7663fff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c15d6c0 0x7f767c14a010 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f767c0647e0 tx=0x7f766c007620 comp rx=0 tx=0).stop 2024-05-08T23:39:51.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.751+0000 7f7682162700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 0x7f767c1f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.751+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c075460 con 0x7f767c1f30a0 2024-05-08T23:39:51.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.752+0000 7f767b7fe700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 0x7f767c1f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.752+0000 7f767b7fe700 1 -- 172.21.15.33:0/4048125175 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f767c0c9ab0 con 0x7f767c1f30a0 2024-05-08T23:39:51.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.752+0000 7f767b7fe700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 0x7f767c1f3480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f766c0017a0 tx=0x7f766c007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c022070 con 0x7f767c1f30a0 2024-05-08T23:39:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4967 (secure 0 0 0) 0x7f766c01d280 con 0x7f767c1f30a0 2024-05-08T23:39:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 msgr2=0x7f767c1f3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 0x7f767c1f3480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f766c0017a0 tx=0x7f766c007980 comp rx=0 tx=0).stop 2024-05-08T23:39:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 msgr2=0x7f767c1f3480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:39:51.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.754+0000 7f7663fff700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f767c1f30a0 0x7f767c1f3480 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f766c0017a0 tx=0x7f766c007980 comp rx=0 tx=0).stop 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 1, 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:39:51.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:39:51.757 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:39:51.760 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:39:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.763 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.765 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:39:51.766 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:39:51.767 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f766806ac90 msgr2=0x7f766806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f7682162700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f766806ac90 0x7f766806d150 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f767c04f3a0 tx=0x7f7670006cf0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 msgr2=0x7f767c149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f7682162700 1 --2- 172.21.15.33:0/4048125175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f767c0c56e0 0x7f767c149ad0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f767c070040 tx=0x7f7674004be0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 reap_dead start 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.755+0000 7f767bfff700 1 -- 172.21.15.33:0/4048125175 reap_dead start 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.756+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 shutdown_connections 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.756+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 >> 172.21.15.33:0/4048125175 conn(0x7f767c06a990 msgr2=0x7f767c151450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.758+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 shutdown_connections 2024-05-08T23:39:51.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.758+0000 7f7682162700 1 -- 172.21.15.33:0/4048125175 wait complete. 2024-05-08T23:39:51.771 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:39:51.771 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:39:51.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:39:51.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.960+0000 7f48d31b5700 1 Processor -- start 2024-05-08T23:39:51.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.960+0000 7f48d31b5700 1 -- start start 2024-05-08T23:39:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d31b5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d31b5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f48cc071700 0x7f48cc15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d31b5700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc0c79f0 con 0x7f48cc15bb30 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d31b5700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc158a90 con 0x7f48cc071700 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d31b5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc06faa0 con 0x7f48cc071320 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f48cc071700 0x7f48cc15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f48cc071700 0x7f48cc15b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:45680/0 (socket says 172.21.15.33:45680) 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 -- 172.21.15.33:0/3189670495 learned_addr learned my addr 172.21.15.33:0/3189670495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48d0f51700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 -- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 msgr2=0x7f48cc06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 -- 172.21.15.33:0/3189670495 >> v1:172.21.15.42:6789/0 conn(0x7f48cc15bb30 legacy=0x7f48cc151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cbfff700 1 -- 172.21.15.33:0/3189670495 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc06b8b0 con 0x7f48cc071700 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.961+0000 7f48cb7fe700 1 -- 172.21.15.33:0/3189670495 <== mon.1 v1:172.21.15.42:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168473885 0 0) 0x7f48cc0c79f0 con 0x7f48cc15bb30 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48cbfff700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc071700 0x7f48cc15b5f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098f0 tx=0x7f48b4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981a0373c74bc318 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48cb7fe700 1 -- 172.21.15.33:0/3189670495 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4011830 con 0x7f48cc071700 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48cb7fe700 1 -- 172.21.15.33:0/3189670495 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b4011990 con 0x7f48cc071700 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48cb7fe700 1 -- 172.21.15.33:0/3189670495 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4011d20 con 0x7f48cc071700 2024-05-08T23:39:51.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 -- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc071700 msgr2=0x7f48cc15b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc071700 0x7f48cc15b5f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48b40098f0 tx=0x7f48b4009c00 comp rx=0 tx=0).stop 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 -- 172.21.15.33:0/3189670495 shutdown_connections 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3189670495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc071700 0x7f48cc15b5f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.962+0000 7f48d31b5700 1 -- 172.21.15.33:0/3189670495 >> 172.21.15.33:0/3189670495 conn(0x7f48cc06e990 msgr2=0x7f48cc06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.963+0000 7f48d31b5700 1 -- 172.21.15.33:0/3189670495 shutdown_connections 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.963+0000 7f48d31b5700 1 -- 172.21.15.33:0/3189670495 wait complete. 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.964+0000 7f48d31b5700 1 Processor -- start 2024-05-08T23:39:51.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.964+0000 7f48d31b5700 1 -- start start 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc15bb30 0x7f48cc1f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc1ed5f0 con 0x7f48cc071700 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc1ee5c0 con 0x7f48cc15bb30 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d31b5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0c8ef0 con 0x7f48cc071320 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d0f51700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46102/0 (socket says 172.21.15.33:46102) 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 -- 172.21.15.33:0/3802021474 learned_addr learned my addr 172.21.15.33:0/3802021474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48d1752700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc15bb30 0x7f48cc1f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 -- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 msgr2=0x7f48cc0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 -- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc15bb30 msgr2=0x7f48cc1f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc15bb30 0x7f48cc1f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.965+0000 7f48cbfff700 1 -- 172.21.15.33:0/3802021474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4009550 con 0x7f48cc071700 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48cbfff700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48cc15c710 tx=0x7f48b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4004300 con 0x7f48cc071700 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b401f070 con 0x7f48cc071700 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc1ed800 con 0x7f48cc071700 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b40036a0 con 0x7f48cc071700 2024-05-08T23:39:51.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.966+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc1eddc0 con 0x7f48cc071700 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.967+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc075460 con 0x7f48cc071700 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.968+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f48b4021030 con 0x7f48cc071700 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.968+0000 7f48c97fa700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 0x7f48ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.968+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f48b408a920 con 0x7f48cc071700 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.968+0000 7f48d0f51700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 0x7f48ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:39:51.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.969+0000 7f48d0f51700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 0x7f48ac06d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48bc010680 tx=0x7f48bc012040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:39:51.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:51.978+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f48b4055ad0 con 0x7f48cc071700 2024-05-08T23:39:52.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.376+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f48cc0c77d0 con 0x7f48cc071700 2024-05-08T23:39:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.378+0000 7f48c97fa700 1 -- 172.21.15.33:0/3802021474 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f48b4015020 con 0x7f48cc071700 2024-05-08T23:39:52.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.400+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 msgr2=0x7f48ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:52.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.400+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 0x7f48ac06d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48bc010680 tx=0x7f48bc012040 comp rx=0 tx=0).stop 2024-05-08T23:39:52.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.400+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 msgr2=0x7f48cc153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:39:52.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.400+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f48cc15c710 tx=0x7f48b4004750 comp rx=0 tx=0).stop 2024-05-08T23:39:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 shutdown_connections 2024-05-08T23:39:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f48ac06ac90 0x7f48ac06d150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48cc15bb30 0x7f48cc1f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f48cc071320 0x7f48cc0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 --2- 172.21.15.33:0/3802021474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc071700 0x7f48cc153b70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.403+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 >> 172.21.15.33:0/3802021474 conn(0x7f48cc06e990 msgr2=0x7f48cc06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.404+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 shutdown_connections 2024-05-08T23:39:52.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:39:52.404+0000 7f48d31b5700 1 -- 172.21.15.33:0/3802021474 wait complete. 2024-05-08T23:39:52.434 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:40:02.436 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:40:02.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.212+0000 7fd07c494700 1 Processor -- start 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.213+0000 7fd07c494700 1 -- start start 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.214+0000 7fd07c494700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0741556e0 0x7fd07406b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.214+0000 7fd07c494700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd0740c7ae0 0x7fd074070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.214+0000 7fd07c494700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740c5dc0 con 0x7fd0740c9050 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.214+0000 7fd07c494700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd074162c90 con 0x7fd0741556e0 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.214+0000 7fd07c494700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07406b180 con 0x7fd0740c7ae0 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0741556e0 0x7fd07406b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd0741556e0 0x7fd07406b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54596/0 (socket says 172.21.15.33:54596) 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 -- 172.21.15.33:0/1225484451 learned_addr learned my addr 172.21.15.33:0/1225484451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:03.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 -- 172.21.15.33:0/1225484451 >> v2:172.21.15.42:3300/0 conn(0x7fd0740c7ae0 msgr2=0x7fd074070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 --2- 172.21.15.33:0/1225484451 >> v2:172.21.15.42:3300/0 conn(0x7fd0740c7ae0 0x7fd074070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 -- 172.21.15.33:0/1225484451 >> v1:172.21.15.42:6789/0 conn(0x7fd0740c9050 legacy=0x7fd074070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.215+0000 7fd07a230700 1 -- 172.21.15.33:0/1225484451 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0741629b0 con 0x7fd0741556e0 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.216+0000 7fd07a230700 1 --2- 172.21.15.33:0/1225484451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0741556e0 0x7fd07406b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0740c96f0 tx=0x7fd064009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593a7b7938183fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.216+0000 7fd07922e700 1 -- 172.21.15.33:0/1225484451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064004230 con 0x7fd0741556e0 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.216+0000 7fd07922e700 1 -- 172.21.15.33:0/1225484451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064004390 con 0x7fd0741556e0 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.216+0000 7fd07922e700 1 -- 172.21.15.33:0/1225484451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0640036a0 con 0x7fd0741556e0 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.217+0000 7fd07c494700 1 -- 172.21.15.33:0/1225484451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0741556e0 msgr2=0x7fd07406b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:03.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.217+0000 7fd07c494700 1 --2- 172.21.15.33:0/1225484451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0741556e0 0x7fd07406b610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0740c96f0 tx=0x7fd064009bd0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.218+0000 7fd07c494700 1 -- 172.21.15.33:0/1225484451 shutdown_connections 2024-05-08T23:40:03.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.218+0000 7fd07c494700 1 --2- 172.21.15.33:0/1225484451 >> v2:172.21.15.42:3300/0 conn(0x7fd0740c7ae0 0x7fd074070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.218+0000 7fd07c494700 1 --2- 172.21.15.33:0/1225484451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0741556e0 0x7fd07406b610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.218+0000 7fd07c494700 1 -- 172.21.15.33:0/1225484451 >> 172.21.15.33:0/1225484451 conn(0x7fd0740c8360 msgr2=0x7fd0740c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:03.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.218+0000 7fd07c494700 1 -- 172.21.15.33:0/1225484451 shutdown_connections 2024-05-08T23:40:03.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.219+0000 7fd07c494700 1 -- 172.21.15.33:0/1225484451 wait complete. 2024-05-08T23:40:03.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.221+0000 7fd07c494700 1 Processor -- start 2024-05-08T23:40:03.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.221+0000 7fd07c494700 1 -- start start 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd0740c7ae0 0x7fd0740caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0741556e0 0x7fd07413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07413d590 con 0x7fd0740c9050 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07413c9c0 con 0x7fd0741556e0 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.222+0000 7fd07c494700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07413cc60 con 0x7fd0740c7ae0 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.223+0000 7fd079a2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54598/0 (socket says 172.21.15.33:54598) 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 -- 172.21.15.33:0/1856250294 learned_addr learned my addr 172.21.15.33:0/1856250294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 -- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd0740c7ae0 msgr2=0x7fd0740caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd0740c7ae0 0x7fd0740caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 -- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0741556e0 msgr2=0x7fd07413f1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0741556e0 0x7fd07413f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.224+0000 7fd079a2f700 1 -- 172.21.15.33:0/1856250294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064009550 con 0x7fd0740c9050 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.225+0000 7fd079a2f700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c4a0 tx=0x7fd07000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.226+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0700054e0 con 0x7fd0740c9050 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.226+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070003680 con 0x7fd0740c9050 2024-05-08T23:40:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.226+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070003a30 con 0x7fd0740c9050 2024-05-08T23:40:03.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.226+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07413cf40 con 0x7fd0740c9050 2024-05-08T23:40:03.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.226+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0740cb100 con 0x7fd0740c9050 2024-05-08T23:40:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.231+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd070003b90 con 0x7fd0740c9050 2024-05-08T23:40:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.232+0000 7fd06affd700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 0x7fd05806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.232+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd070088a20 con 0x7fd0740c9050 2024-05-08T23:40:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.234+0000 7fd07a230700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 0x7fd05806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.236+0000 7fd07a230700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 0x7fd05806d220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:03.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.306+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c0052f0 con 0x7fd0740c9050 2024-05-08T23:40:03.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:03.322+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd070053a80 con 0x7fd0740c9050 2024-05-08T23:40:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.131+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd05c005b20 con 0x7fd0740c9050 2024-05-08T23:40:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.143+0000 7fd06affd700 1 -- 172.21.15.33:0/1856250294 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fd070017030 con 0x7fd0740c9050 2024-05-08T23:40:04.175 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.198+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 msgr2=0x7fd05806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.198+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 0x7fd05806d220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064010000 comp rx=0 tx=0).stop 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.198+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 msgr2=0x7fd0741369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.198+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c4a0 tx=0x7fd07000c860 comp rx=0 tx=0).stop 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 shutdown_connections 2024-05-08T23:40:04.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd05806ad60 0x7fd05806d220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0741556e0 0x7fd07413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd0740c7ae0 0x7fd0740caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 --2- 172.21.15.33:0/1856250294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0740c9050 0x7fd0741369a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.199+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 >> 172.21.15.33:0/1856250294 conn(0x7fd0740c8360 msgr2=0x7fd074070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.200+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 shutdown_connections 2024-05-08T23:40:04.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.200+0000 7fd07c494700 1 -- 172.21.15.33:0/1856250294 wait complete. 2024-05-08T23:40:04.345 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:40:04.345 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T23:40:04.345 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:40:04.345 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:40:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.956+0000 7f91fa3e6700 1 Processor -- start 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.957+0000 7f91fa3e6700 1 -- start start 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91fa3e6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f40c70c0 0x7f91f40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91fa3e6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91f41577a0 0x7f91f4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91fa3e6700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f41572f0 con 0x7f91f4150510 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91fa3e6700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f4157c30 con 0x7f91f41577a0 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91fa3e6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40c8110 con 0x7f91f40c70c0 2024-05-08T23:40:04.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91f8983700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91f41577a0 0x7f91f4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91f8983700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f91f41577a0 0x7f91f4156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54612/0 (socket says 172.21.15.33:54612) 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.959+0000 7f91f8983700 1 -- 172.21.15.33:0/3790109062 learned_addr learned my addr 172.21.15.33:0/3790109062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.960+0000 7f91f8983700 1 -- 172.21.15.33:0/3790109062 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f40c70c0 msgr2=0x7f91f40c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.960+0000 7f91f8983700 1 --2- 172.21.15.33:0/3790109062 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f40c70c0 0x7f91f40c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.960+0000 7f91f8983700 1 -- 172.21.15.33:0/3790109062 >> v1:172.21.15.42:6789/0 conn(0x7f91f4150510 legacy=0x7f91f406a990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:40:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.960+0000 7f91f8983700 1 -- 172.21.15.33:0/3790109062 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40c7e30 con 0x7f91f41577a0 2024-05-08T23:40:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.961+0000 7f91f8983700 1 --2- 172.21.15.33:0/3790109062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f41577a0 0x7f91f4156db0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009bd0 tx=0x7f91ec00d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee619f40dfeb59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.962+0000 7f91f2ffd700 1 -- 172.21.15.33:0/3790109062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00dcd0 con 0x7f91f41577a0 2024-05-08T23:40:04.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.962+0000 7f91f2ffd700 1 -- 172.21.15.33:0/3790109062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec00de30 con 0x7f91f41577a0 2024-05-08T23:40:04.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.962+0000 7f91f2ffd700 1 -- 172.21.15.33:0/3790109062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec003c60 con 0x7f91f41577a0 2024-05-08T23:40:04.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.963+0000 7f91fa3e6700 1 -- 172.21.15.33:0/3790109062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f41577a0 msgr2=0x7f91f4156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:04.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.963+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/3790109062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f41577a0 0x7f91f4156db0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009bd0 tx=0x7f91ec00d3c0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.965+0000 7f91fa3e6700 1 -- 172.21.15.33:0/3790109062 shutdown_connections 2024-05-08T23:40:04.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.965+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/3790109062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f41577a0 0x7f91f4156db0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.965+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/3790109062 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f40c70c0 0x7f91f40c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:04.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.965+0000 7f91fa3e6700 1 -- 172.21.15.33:0/3790109062 >> 172.21.15.33:0/3790109062 conn(0x7f91f40c8360 msgr2=0x7f91f40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.991+0000 7f91fa3e6700 1 -- 172.21.15.33:0/3790109062 shutdown_connections 2024-05-08T23:40:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.992+0000 7f91fa3e6700 1 -- 172.21.15.33:0/3790109062 wait complete. 2024-05-08T23:40:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.994+0000 7f91fa3e6700 1 Processor -- start 2024-05-08T23:40:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.995+0000 7f91fa3e6700 1 -- start start 2024-05-08T23:40:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f41577a0 0x7f91f4147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 0x7f91f41f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f41f7800 0x7f91f40c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41f4290 con 0x7f91f4071bb0 2024-05-08T23:40:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41f7fd0 con 0x7f91f41577a0 2024-05-08T23:40:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.996+0000 7f91fa3e6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41f82b0 con 0x7f91f41f7800 2024-05-08T23:40:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.998+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 0x7f91f41f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.998+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 0x7f91f41f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54620/0 (socket says 172.21.15.33:54620) 2024-05-08T23:40:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:04.998+0000 7f91f37fe700 1 -- 172.21.15.33:0/4084476520 learned_addr learned my addr 172.21.15.33:0/4084476520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.003+0000 7f91f37fe700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f41f7800 msgr2=0x7f91f40c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.003+0000 7f91f37fe700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91f41f7800 0x7f91f40c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.003+0000 7f91f37fe700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f41577a0 msgr2=0x7f91f4147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.003+0000 7f91f37fe700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f41577a0 0x7f91f4147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.003+0000 7f91f37fe700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec009830 con 0x7f91f4071bb0 2024-05-08T23:40:05.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.010+0000 7f91f37fe700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 0x7f91f41f4d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91e800b500 tx=0x7f91e800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.028+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8011830 con 0x7f91f4071bb0 2024-05-08T23:40:05.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.028+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8011990 con 0x7f91f4071bb0 2024-05-08T23:40:05.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.028+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f41f8590 con 0x7f91f4071bb0 2024-05-08T23:40:05.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.028+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f41f4830 con 0x7f91f4071bb0 2024-05-08T23:40:05.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.029+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8011d40 con 0x7f91f4071bb0 2024-05-08T23:40:05.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.033+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f91e801a410 con 0x7f91f4071bb0 2024-05-08T23:40:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.033+0000 7f91f0ff9700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91dc06aea0 0x7f91dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.033+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f91e8088de0 con 0x7f91f4071bb0 2024-05-08T23:40:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.044+0000 7f91f3fff700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91dc06aea0 0x7f91dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.045+0000 7f91f3fff700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91dc06aea0 0x7f91dc06d360 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91f4067400 tx=0x7f91e4006cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.096+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 0x7f91d4004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.096+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4007f20 con 0x7f91d4002a40 2024-05-08T23:40:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.097+0000 7f91f8983700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 0x7f91d4004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.098+0000 7f91f8983700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91d40080e0 con 0x7f91d4002a40 2024-05-08T23:40:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.098+0000 7f91f8983700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 0x7f91d4004f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91ec016000 tx=0x7f91ec004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec022070 con 0x7f91d4002a40 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f91ec0092e0 con 0x7f91d4002a40 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 msgr2=0x7f91d4004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 0x7f91d4004f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91ec016000 tx=0x7f91ec004310 comp rx=0 tx=0).stop 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 msgr2=0x7f91d4004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.102+0000 7f91f0ff9700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4002a40 0x7f91d4004f00 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91ec016000 tx=0x7f91ec004310 comp rx=0 tx=0).stop 2024-05-08T23:40:05.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.143+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 0x7f91d4007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.144+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d400a3e0 con 0x7f91d4006fc0 2024-05-08T23:40:05.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.144+0000 7f91f8983700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 0x7f91d4007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.145+0000 7f91f8983700 1 -- 172.21.15.33:0/4084476520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91d400a5a0 con 0x7f91d4006fc0 2024-05-08T23:40:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.145+0000 7f91f8983700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 0x7f91d4007450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01f000 tx=0x7f91ec009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec003e80 con 0x7f91d4006fc0 2024-05-08T23:40:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4460 (secure 0 0 0) 0x7f91ec004940 con 0x7f91d4006fc0 2024-05-08T23:40:05.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 msgr2=0x7f91d4007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 0x7f91d4007450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01f000 tx=0x7f91ec009000 comp rx=0 tx=0).stop 2024-05-08T23:40:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 msgr2=0x7f91d4007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.146+0000 7f91f0ff9700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91d4006fc0 0x7f91d4007450 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01f000 tx=0x7f91ec009000 comp rx=0 tx=0).stop 2024-05-08T23:40:05.145 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:40:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 14, 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.154 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.155 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:05.159 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.204+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91dc06aea0 msgr2=0x7f91dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.204+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91dc06aea0 0x7f91dc06d360 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91f4067400 tx=0x7f91e4006cd0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.204+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 msgr2=0x7f91f41f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.204+0000 7f91fa3e6700 1 --2- 172.21.15.33:0/4084476520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f4071bb0 0x7f91f41f4d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f91e800b500 tx=0x7f91e800b8c0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.204+0000 7f91f8983700 1 -- 172.21.15.33:0/4084476520 reap_dead start 2024-05-08T23:40:05.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.205+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 shutdown_connections 2024-05-08T23:40:05.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.205+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 >> 172.21.15.33:0/4084476520 conn(0x7f91f40c8360 msgr2=0x7f91f4156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:05.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.214+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 shutdown_connections 2024-05-08T23:40:05.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.214+0000 7f91fa3e6700 1 -- 172.21.15.33:0/4084476520 wait complete. 2024-05-08T23:40:05.260 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:05.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:40:05.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:40:05.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.774+0000 7f8134e2f700 1 Processor -- start 2024-05-08T23:40:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.774+0000 7f8134e2f700 1 -- start start 2024-05-08T23:40:05.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f8134e2f700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8130150510 0x7f81300c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f8134e2f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f8134e2f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81301572f0 con 0x7f81300c70c0 2024-05-08T23:40:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f8134e2f700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8130157c30 con 0x7f8130150510 2024-05-08T23:40:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f8134e2f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81300c8110 con 0x7f81301577a0 2024-05-08T23:40:05.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f812ed9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f812ed9d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:45630/0 (socket says 172.21.15.33:45630) 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.776+0000 7f812ed9d700 1 -- 172.21.15.33:0/2386267693 learned_addr learned my addr 172.21.15.33:0/2386267693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.777+0000 7f812ed9d700 1 -- 172.21.15.33:0/2386267693 >> v2:172.21.15.33:3300/0 conn(0x7f8130150510 msgr2=0x7f81300c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.777+0000 7f812ed9d700 1 --2- 172.21.15.33:0/2386267693 >> v2:172.21.15.33:3300/0 conn(0x7f8130150510 0x7f81300c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.777+0000 7f812ed9d700 1 -- 172.21.15.33:0/2386267693 >> v1:172.21.15.33:6789/0 conn(0x7f81300c70c0 legacy=0x7f813006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:05.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.777+0000 7f812ed9d700 1 -- 172.21.15.33:0/2386267693 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300c7e30 con 0x7f81301577a0 2024-05-08T23:40:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.777+0000 7f812ed9d700 1 --2- 172.21.15.33:0/2386267693 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8128009bd0 tx=0x7f812800d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a96f3312e1666cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.778+0000 7f812d59a700 1 -- 172.21.15.33:0/2386267693 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800dcd0 con 0x7f81301577a0 2024-05-08T23:40:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.778+0000 7f812d59a700 1 -- 172.21.15.33:0/2386267693 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800de30 con 0x7f81301577a0 2024-05-08T23:40:05.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.778+0000 7f812d59a700 1 -- 172.21.15.33:0/2386267693 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128003c40 con 0x7f81301577a0 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 -- 172.21.15.33:0/2386267693 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 msgr2=0x7f8130156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 --2- 172.21.15.33:0/2386267693 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8128009bd0 tx=0x7f812800d3c0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 -- 172.21.15.33:0/2386267693 shutdown_connections 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 --2- 172.21.15.33:0/2386267693 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f8130156db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 --2- 172.21.15.33:0/2386267693 >> v2:172.21.15.33:3300/0 conn(0x7f8130150510 0x7f81300c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 -- 172.21.15.33:0/2386267693 >> 172.21.15.33:0/2386267693 conn(0x7f81300c8360 msgr2=0x7f81300c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:05.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.779+0000 7f8134e2f700 1 -- 172.21.15.33:0/2386267693 shutdown_connections 2024-05-08T23:40:05.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.780+0000 7f8134e2f700 1 -- 172.21.15.33:0/2386267693 wait complete. 2024-05-08T23:40:05.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.782+0000 7f8134e2f700 1 Processor -- start 2024-05-08T23:40:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.782+0000 7f8134e2f700 1 -- start start 2024-05-08T23:40:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81300c70c0 0x7f81300c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 0x7f813014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f813014b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813014fb50 con 0x7f8130150510 2024-05-08T23:40:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81301f5070 con 0x7f81300c70c0 2024-05-08T23:40:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.783+0000 7f8134e2f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81301f41e0 con 0x7f81301577a0 2024-05-08T23:40:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.784+0000 7f812dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 0x7f813014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.784+0000 7f812dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 0x7f813014b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39698/0 (socket says 172.21.15.33:39698) 2024-05-08T23:40:05.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.784+0000 7f812dd9b700 1 -- 172.21.15.33:0/3749608827 learned_addr learned my addr 172.21.15.33:0/3749608827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:05.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f813014b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812dd9b700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 msgr2=0x7f813014b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812dd9b700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f813014b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812dd9b700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81300c70c0 msgr2=0x7f81300c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812dd9b700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81300c70c0 0x7f81300c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.785+0000 7f812dd9b700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128009830 con 0x7f8130150510 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.786+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f81301577a0 0x7f813014b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:40:05.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.786+0000 7f812dd9b700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 0x7f813014b230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f812400b470 tx=0x7f812400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.801+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011830 con 0x7f8130150510 2024-05-08T23:40:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.801+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81240119b0 con 0x7f8130150510 2024-05-08T23:40:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.801+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124011d60 con 0x7f8130150510 2024-05-08T23:40:05.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.801+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81301f4460 con 0x7f8130150510 2024-05-08T23:40:05.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.801+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81301f4a20 con 0x7f8130150510 2024-05-08T23:40:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.804+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f812401a400 con 0x7f8130150510 2024-05-08T23:40:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.805+0000 7f811effd700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811806ae70 0x7f811806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.805+0000 7f812e59c700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811806ae70 0x7f811806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.806+0000 7f812e59c700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811806ae70 0x7f811806d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8120009710 tx=0x7f8120006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.810+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f81240588c0 con 0x7f8130150510 2024-05-08T23:40:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.813+0000 7f8134e2f700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 0x7f8114004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.813+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8114007f20 con 0x7f8114002a40 2024-05-08T23:40:05.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.816+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 0x7f8114004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.817+0000 7f812ed9d700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81140080e0 con 0x7f8114002a40 2024-05-08T23:40:05.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.817+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 0x7f8114004f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8128009bd0 tx=0x7f8128004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280042b0 con 0x7f8114002a40 2024-05-08T23:40:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8128004410 con 0x7f8114002a40 2024-05-08T23:40:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 msgr2=0x7f8114004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 0x7f8114004f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8128009bd0 tx=0x7f8128004ad0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 msgr2=0x7f8114004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.825+0000 7f811effd700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114002a40 0x7f8114004f00 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8128009bd0 tx=0x7f8128004ad0 comp rx=0 tx=0).stop 2024-05-08T23:40:05.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.855+0000 7f8134e2f700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 0x7f8114007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:05.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.855+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811400a3e0 con 0x7f8114006fc0 2024-05-08T23:40:05.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.859+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 0x7f8114007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:05.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.860+0000 7f812ed9d700 1 -- 172.21.15.33:0/3749608827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f811400a5a0 con 0x7f8114006fc0 2024-05-08T23:40:05.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.860+0000 7f812ed9d700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 0x7f8114007450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81140080b0 tx=0x7f8128009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812802f070 con 0x7f8114006fc0 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4941 (secure 0 0 0) 0x7f812800e640 con 0x7f8114006fc0 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 msgr2=0x7f8114007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 0x7f8114007450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81140080b0 tx=0x7f8128009000 comp rx=0 tx=0).stop 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 msgr2=0x7f8114007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:05.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.868+0000 7f811effd700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114006fc0 0x7f8114007450 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81140080b0 tx=0x7f8128009000 comp rx=0 tx=0).stop 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 15, 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:40:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:05.881 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:05.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.892+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811806ae70 msgr2=0x7f811806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.892+0000 7f8134e2f700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f811806ae70 0x7f811806d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8120009710 tx=0x7f8120006c80 comp rx=0 tx=0).stop 2024-05-08T23:40:05.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.892+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 msgr2=0x7f813014b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:05.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.892+0000 7f8134e2f700 1 --2- 172.21.15.33:0/3749608827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8130150510 0x7f813014b230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f812400b470 tx=0x7f812400b830 comp rx=0 tx=0).stop 2024-05-08T23:40:05.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.892+0000 7f812ed9d700 1 -- 172.21.15.33:0/3749608827 reap_dead start 2024-05-08T23:40:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.898+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 shutdown_connections 2024-05-08T23:40:05.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.898+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 >> 172.21.15.33:0/3749608827 conn(0x7f81300c8360 msgr2=0x7f813014e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.898+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 shutdown_connections 2024-05-08T23:40:05.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:05.898+0000 7f8134e2f700 1 -- 172.21.15.33:0/3749608827 wait complete. 2024-05-08T23:40:05.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:05.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:40:05.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:40:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.473+0000 7fb908ce5700 1 Processor -- start 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.473+0000 7fb908ce5700 1 -- start start 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb904150510 0x7fb9040c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb9040c70c0 0x7fb9041577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fb904156db0 0x7fb90406a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040c8110 con 0x7fb904150510 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040c7e30 con 0x7fb904156db0 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb908ce5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040c6300 con 0x7fb9040c70c0 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb904150510 0x7fb9040c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb904150510 0x7fb9040c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39706/0 (socket says 172.21.15.33:39706) 2024-05-08T23:40:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 -- 172.21.15.33:0/131842130 learned_addr learned my addr 172.21.15.33:0/131842130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:06.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 -- 172.21.15.33:0/131842130 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb9040c70c0 msgr2=0x7fb9041577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:06.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 --2- 172.21.15.33:0/131842130 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb9040c70c0 0x7fb9041577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 -- 172.21.15.33:0/131842130 >> v2:172.21.15.42:3300/0 conn(0x7fb904156db0 msgr2=0x7fb90406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:06.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.475+0000 7fb90259c700 1 --2- 172.21.15.33:0/131842130 >> v2:172.21.15.42:3300/0 conn(0x7fb904156db0 0x7fb90406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.476+0000 7fb90259c700 1 -- 172.21.15.33:0/131842130 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040c65e0 con 0x7fb904150510 2024-05-08T23:40:06.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.492+0000 7fb90259c700 1 --2- 172.21.15.33:0/131842130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904150510 0x7fb9040c7780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb90406b4b0 tx=0x7fb8f40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265425aa23998bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.494+0000 7fb90159a700 1 -- 172.21.15.33:0/131842130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400c980 con 0x7fb904150510 2024-05-08T23:40:06.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.494+0000 7fb90159a700 1 -- 172.21.15.33:0/131842130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400cae0 con 0x7fb904150510 2024-05-08T23:40:06.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.494+0000 7fb90159a700 1 -- 172.21.15.33:0/131842130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400ce70 con 0x7fb904150510 2024-05-08T23:40:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.516+0000 7fb908ce5700 1 -- 172.21.15.33:0/131842130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904150510 msgr2=0x7fb9040c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.516+0000 7fb908ce5700 1 --2- 172.21.15.33:0/131842130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904150510 0x7fb9040c7780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb90406b4b0 tx=0x7fb8f40095c0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 -- 172.21.15.33:0/131842130 shutdown_connections 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 --2- 172.21.15.33:0/131842130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb904150510 0x7fb9040c7780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 --2- 172.21.15.33:0/131842130 >> v2:172.21.15.42:3300/0 conn(0x7fb904156db0 0x7fb90406a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 --2- 172.21.15.33:0/131842130 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb9040c70c0 0x7fb9041577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 -- 172.21.15.33:0/131842130 >> 172.21.15.33:0/131842130 conn(0x7fb9040c8360 msgr2=0x7fb9040c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 -- 172.21.15.33:0/131842130 shutdown_connections 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.551+0000 7fb908ce5700 1 -- 172.21.15.33:0/131842130 wait complete. 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.553+0000 7fb908ce5700 1 Processor -- start 2024-05-08T23:40:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.554+0000 7fb908ce5700 1 -- start start 2024-05-08T23:40:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 0x7fb9041520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb904150510 0x7fb9040c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb904156db0 0x7fb90414ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9041525e0 con 0x7fb9040c70c0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb904153a10 con 0x7fb904156db0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb908ce5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90420e9e0 con 0x7fb904150510 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb90259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 0x7fb9041520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb90259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 0x7fb9041520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39722/0 (socket says 172.21.15.33:39722) 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.557+0000 7fb90259c700 1 -- 172.21.15.33:0/3353244685 learned_addr learned my addr 172.21.15.33:0/3353244685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb904150510 msgr2=0x7fb9040c4790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb904150510 0x7fb9040c4790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb904156db0 msgr2=0x7fb90414ec80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb904156db0 0x7fb90414ec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4009de0 con 0x7fb9040c70c0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.558+0000 7fb90259c700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 0x7fb9041520a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb90406f340 tx=0x7fb8f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.559+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4003ab0 con 0x7fb9040c70c0 2024-05-08T23:40:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.559+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f401d040 con 0x7fb9040c70c0 2024-05-08T23:40:06.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.559+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4003e30 con 0x7fb9040c70c0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.559+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9040c5b20 con 0x7fb9040c70c0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.559+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90420ef10 con 0x7fb9040c70c0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.562+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fb8f4018400 con 0x7fb9040c70c0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.563+0000 7fb8f2ffd700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb8ec06aef0 0x7fb8ec06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.563+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fb8f408a8f0 con 0x7fb9040c70c0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.563+0000 7fb901d9b700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb8ec06aef0 0x7fb8ec06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.564+0000 7fb901d9b700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb8ec06aef0 0x7fb8ec06d3b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9040c5d10 tx=0x7fb8f8006c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.574+0000 7fb908ce5700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 0x7fb8e8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.574+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8007f20 con 0x7fb8e8002a40 2024-05-08T23:40:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.579+0000 7fb902d9d700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 0x7fb8e8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.584+0000 7fb902d9d700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8e80080e0 con 0x7fb8e8002a40 2024-05-08T23:40:06.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.588+0000 7fb902d9d700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 0x7fb8e8004f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.592+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00cb30 con 0x7fb8e8002a40 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.593+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb8fc00cc90 con 0x7fb8e8002a40 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.593+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 msgr2=0x7fb8e8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.593+0000 7fb8f2ffd700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 0x7fb8e8004f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc009e50 comp rx=0 tx=0).stop 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.593+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 msgr2=0x7fb8e8004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.593+0000 7fb8f2ffd700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8002a40 0x7fb8e8004f00 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc009e50 comp rx=0 tx=0).stop 2024-05-08T23:40:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.646+0000 7fb908ce5700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 0x7fb8e8007450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.646+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800a3e0 con 0x7fb8e8006fc0 2024-05-08T23:40:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.650+0000 7fb902d9d700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 0x7fb8e8007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.651+0000 7fb902d9d700 1 -- 172.21.15.33:0/3353244685 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8e800a5a0 con 0x7fb8e8006fc0 2024-05-08T23:40:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.651+0000 7fb902d9d700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 0x7fb8e8007450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:06.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc01c8e0 con 0x7fb8e8006fc0 2024-05-08T23:40:06.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4968 (secure 0 0 0) 0x7fb8fc020070 con 0x7fb8e8006fc0 2024-05-08T23:40:06.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 msgr2=0x7fb8e8007450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 0x7fb8e8007450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 msgr2=0x7fb8e8007450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:06.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.659+0000 7fb8f2ffd700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fb8e8006fc0 0x7fb8e8007450 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00bd50 tx=0x7fb8fc004ad0 comp rx=0 tx=0).stop 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 64, 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 16, 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:40:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:06.693 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.733+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb8ec06aef0 msgr2=0x7fb8ec06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.733+0000 7fb908ce5700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fb8ec06aef0 0x7fb8ec06d3b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9040c5d10 tx=0x7fb8f8006c10 comp rx=0 tx=0).stop 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.733+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 msgr2=0x7fb9041520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.733+0000 7fb908ce5700 1 --2- 172.21.15.33:0/3353244685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9040c70c0 0x7fb9041520a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb90406f340 tx=0x7fb8f4004750 comp rx=0 tx=0).stop 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.734+0000 7fb902d9d700 1 -- 172.21.15.33:0/3353244685 reap_dead start 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.734+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 shutdown_connections 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.734+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 >> 172.21.15.33:0/3353244685 conn(0x7fb9040c8360 msgr2=0x7fb9040c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.734+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 shutdown_connections 2024-05-08T23:40:06.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:06.734+0000 7fb908ce5700 1 -- 172.21.15.33:0/3353244685 wait complete. 2024-05-08T23:40:06.780 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:06.781 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-08T23:40:06.781 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T23:40:06.781 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-08T23:40:06.781 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:40:06.781 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:40:06.781 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:40:06.804 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:40:06.804 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-08T23:40:06.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.334+0000 7f4a50324700 1 Processor -- start 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.334+0000 7f4a50324700 1 -- start start 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4a48150510 0x7f4a480c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a480c70c0 0x7f4a481577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 0x7f4a4806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480c8110 con 0x7f4a48156db0 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480c7e30 con 0x7f4a48150510 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a50324700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480c6300 con 0x7f4a480c70c0 2024-05-08T23:40:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a4e8c1700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 msgr2=0x7f4a4806a990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.336+0000 7f4a4e8c1700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 0x7f4a4806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.337+0000 7f4a4e0c0700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4a48150510 0x7f4a480c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.337+0000 7f4a4e0c0700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f4a48150510 0x7f4a480c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46598/0 (socket says 172.21.15.33:46598) 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.337+0000 7f4a4e0c0700 1 -- 172.21.15.33:0/70705144 learned_addr learned my addr 172.21.15.33:0/70705144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 -- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a480c70c0 msgr2=0x7f4a481577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 --2- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a480c70c0 0x7f4a481577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 -- 172.21.15.33:0/70705144 >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 msgr2=0x7f4a4806a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 --2- 172.21.15.33:0/70705144 >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 0x7f4a4806a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 -- 172.21.15.33:0/70705144 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480c65e0 con 0x7f4a48150510 2024-05-08T23:40:07.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.338+0000 7f4a4e0c0700 1 --2- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c7780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4806f310 tx=0x7f4a44010730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb7a854c83c93de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.346+0000 7f4a4d0be700 1 -- 172.21.15.33:0/70705144 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400ada0 con 0x7f4a48150510 2024-05-08T23:40:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.347+0000 7f4a4d0be700 1 -- 172.21.15.33:0/70705144 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4400b050 con 0x7f4a48150510 2024-05-08T23:40:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.347+0000 7f4a4d0be700 1 -- 172.21.15.33:0/70705144 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400b3e0 con 0x7f4a48150510 2024-05-08T23:40:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.348+0000 7f4a50324700 1 -- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 msgr2=0x7f4a480c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.348+0000 7f4a50324700 1 --2- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c7780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a4806f310 tx=0x7f4a44010730 comp rx=0 tx=0).stop 2024-05-08T23:40:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.349+0000 7f4a50324700 1 -- 172.21.15.33:0/70705144 shutdown_connections 2024-05-08T23:40:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.349+0000 7f4a50324700 1 --2- 172.21.15.33:0/70705144 >> v2:172.21.15.33:3300/0 conn(0x7f4a48156db0 0x7f4a4806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.349+0000 7f4a50324700 1 --2- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c7780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.349+0000 7f4a50324700 1 --2- 172.21.15.33:0/70705144 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a480c70c0 0x7f4a481577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.349+0000 7f4a50324700 1 -- 172.21.15.33:0/70705144 >> 172.21.15.33:0/70705144 conn(0x7f4a480c8360 msgr2=0x7f4a480c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:07.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.354+0000 7f4a50324700 1 -- 172.21.15.33:0/70705144 shutdown_connections 2024-05-08T23:40:07.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.355+0000 7f4a50324700 1 -- 172.21.15.33:0/70705144 wait complete. 2024-05-08T23:40:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.367+0000 7f4a50324700 1 Processor -- start 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.368+0000 7f4a50324700 1 -- start start 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 0x7f4a481550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 0x7f4a481ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481efcc0 con 0x7f4a480c70c0 2024-05-08T23:40:07.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48155630 con 0x7f4a48150510 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a50324700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a481558c0 con 0x7f4a48156db0 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4e0c0700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 msgr2=0x7f4a481550f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4e0c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 0x7f4a481550f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4e8c1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 0x7f4a481ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4e8c1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 0x7f4a481ecc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:45660/0 (socket says 172.21.15.33:45660) 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4d8bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:07.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.369+0000 7f4a4e8c1700 1 -- 172.21.15.33:0/1244110915 learned_addr learned my addr 172.21.15.33:0/1244110915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.375+0000 7f4a4d8bf700 1 -- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 msgr2=0x7f4a481ecc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.375+0000 7f4a4d8bf700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 0x7f4a481ecc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.375+0000 7f4a4d8bf700 1 -- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 msgr2=0x7f4a481550f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.375+0000 7f4a4d8bf700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 0x7f4a481550f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.375+0000 7f4a4d8bf700 1 -- 172.21.15.33:0/1244110915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4400a5f0 con 0x7f4a48150510 2024-05-08T23:40:07.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.389+0000 7f4a4d8bf700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c1960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b410 tx=0x7f4a3800b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:07.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.396+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38011830 con 0x7f4a48150510 2024-05-08T23:40:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.397+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3800be90 con 0x7f4a48150510 2024-05-08T23:40:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.397+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38011bb0 con 0x7f4a48150510 2024-05-08T23:40:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.398+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a481eef00 con 0x7f4a48150510 2024-05-08T23:40:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.398+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a481ef4c0 con 0x7f4a48150510 2024-05-08T23:40:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.402+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c0052f0 con 0x7f4a48150510 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.404+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f4a38011d10 con 0x7f4a48150510 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.404+0000 7f4a3effd700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 0x7f4a3406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.404+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f4a38089330 con 0x7f4a48150510 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.405+0000 7f4a4e0c0700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 0x7f4a3406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.405+0000 7f4a4e0c0700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 0x7f4a3406d290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a4401c040 tx=0x7f4a44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:07.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:07.415+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f4a38054310 con 0x7f4a48150510 2024-05-08T23:40:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.207+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4a30005180 con 0x7f4a48150510 2024-05-08T23:40:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.209+0000 7f4a3effd700 1 -- 172.21.15.33:0/1244110915 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f4a38015080 con 0x7f4a48150510 2024-05-08T23:40:08.252 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.278+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 msgr2=0x7f4a3406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.278+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 0x7f4a3406d290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a4401c040 tx=0x7f4a44008040 comp rx=0 tx=0).stop 2024-05-08T23:40:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.278+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 msgr2=0x7f4a480c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:08.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.278+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c1960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b410 tx=0x7f4a3800b720 comp rx=0 tx=0).stop 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 shutdown_connections 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f4a3406add0 0x7f4a3406d290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f4a48156db0 0x7f4a481ecc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a480c70c0 0x7f4a481550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 --2- 172.21.15.33:0/1244110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a48150510 0x7f4a480c1960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.286+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 >> 172.21.15.33:0/1244110915 conn(0x7f4a480c8360 msgr2=0x7f4a480c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.292+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 shutdown_connections 2024-05-08T23:40:08.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:08.292+0000 7f4a50324700 1 -- 172.21.15.33:0/1244110915 wait complete. 2024-05-08T23:40:11.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:11.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.939+0000 7fd3f7d32700 1 Processor -- start 2024-05-08T23:40:11.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.940+0000 7fd3f7d32700 1 -- start start 2024-05-08T23:40:11.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f7d32700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3f0070800 0x7fd3f0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f7d32700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 0x7fd3f00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f7d32700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00c85a0 con 0x7fd3f00c8c50 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f7d32700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00c9570 con 0x7fd3f00c7330 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f7d32700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f0156b40 con 0x7fd3f0070800 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f5ace700 1 -- >> v1:172.21.15.33:6789/0 conn(0x7fd3f00c8c50 legacy=0x7fd3f0156fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f62cf700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 msgr2=0x7fd3f00c9030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.941+0000 7fd3f62cf700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 0x7fd3f00c9030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:11.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.947+0000 7fd3f52cd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3f0070800 0x7fd3f0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:11.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.947+0000 7fd3f52cd700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3f0070800 0x7fd3f0071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:46618/0 (socket says 172.21.15.33:46618) 2024-05-08T23:40:11.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.947+0000 7fd3f52cd700 1 -- 172.21.15.33:0/1165221030 learned_addr learned my addr 172.21.15.33:0/1165221030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.956+0000 7fd3f52cd700 1 -- 172.21.15.33:0/1165221030 >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 msgr2=0x7fd3f00c9030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.956+0000 7fd3f52cd700 1 --2- 172.21.15.33:0/1165221030 >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 0x7fd3f00c9030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.956+0000 7fd3f52cd700 1 -- 172.21.15.33:0/1165221030 >> v1:172.21.15.33:6789/0 conn(0x7fd3f00c8c50 legacy=0x7fd3f0156fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.956+0000 7fd3f52cd700 1 -- 172.21.15.33:0/1165221030 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00c87d0 con 0x7fd3f0070800 2024-05-08T23:40:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.957+0000 7fd3f52cd700 1 --2- 172.21.15.33:0/1165221030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0070800 0x7fd3f0071bb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00c6880 tx=0x7fd3e000be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb64440679e72e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.957+0000 7fd3f4acc700 1 -- 172.21.15.33:0/1165221030 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000dbe0 con 0x7fd3f0070800 2024-05-08T23:40:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.957+0000 7fd3f4acc700 1 -- 172.21.15.33:0/1165221030 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000dd40 con 0x7fd3f0070800 2024-05-08T23:40:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.957+0000 7fd3f4acc700 1 -- 172.21.15.33:0/1165221030 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0004270 con 0x7fd3f0070800 2024-05-08T23:40:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 -- 172.21.15.33:0/1165221030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0070800 msgr2=0x7fd3f0071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/1165221030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0070800 0x7fd3f0071bb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00c6880 tx=0x7fd3e000be70 comp rx=0 tx=0).stop 2024-05-08T23:40:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 -- 172.21.15.33:0/1165221030 shutdown_connections 2024-05-08T23:40:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/1165221030 >> v2:172.21.15.33:3300/0 conn(0x7fd3f00c7330 0x7fd3f00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/1165221030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f0070800 0x7fd3f0071bb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.958+0000 7fd3f7d32700 1 -- 172.21.15.33:0/1165221030 >> 172.21.15.33:0/1165221030 conn(0x7fd3f006a990 msgr2=0x7fd3f006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.959+0000 7fd3f7d32700 1 -- 172.21.15.33:0/1165221030 shutdown_connections 2024-05-08T23:40:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.959+0000 7fd3f7d32700 1 -- 172.21.15.33:0/1165221030 wait complete. 2024-05-08T23:40:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.961+0000 7fd3f7d32700 1 Processor -- start 2024-05-08T23:40:11.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.971+0000 7fd3f7d32700 1 -- start start 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 0x7fd3f014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00c7330 0x7fd3f0153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f014ca90 con 0x7fd3f0070800 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0152280 con 0x7fd3f00c7330 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.972+0000 7fd3f7d32700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0152500 con 0x7fd3f00c8c50 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.974+0000 7fd3f62cf700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.974+0000 7fd3f62cf700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:45676/0 (socket says 172.21.15.33:45676) 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.974+0000 7fd3f62cf700 1 -- 172.21.15.33:0/4249245849 learned_addr learned my addr 172.21.15.33:0/4249245849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.978+0000 7fd3f5ace700 1 -- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 msgr2=0x7fd3f014ceb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:11.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.978+0000 7fd3f5ace700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 0x7fd3f014ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:11.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.980+0000 7fd3f62cf700 1 -- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00c7330 msgr2=0x7fd3f0153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:11.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.980+0000 7fd3f62cf700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00c7330 0x7fd3f0153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.980+0000 7fd3f62cf700 1 -- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 msgr2=0x7fd3f014ceb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.980+0000 7fd3f62cf700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 0x7fd3f014ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:11.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.980+0000 7fd3f62cf700 1 -- 172.21.15.33:0/4249245849 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e000b7f0 con 0x7fd3f00c8c50 2024-05-08T23:40:11.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.984+0000 7fd3f62cf700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b450 tx=0x7fd3ec00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:11.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.992+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec003e50 con 0x7fd3f00c8c50 2024-05-08T23:40:11.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.992+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec0060d0 con 0x7fd3f00c8c50 2024-05-08T23:40:11.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.992+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec006440 con 0x7fd3f00c8c50 2024-05-08T23:40:11.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.993+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f0159d30 con 0x7fd3f00c8c50 2024-05-08T23:40:11.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.994+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f01511c0 con 0x7fd3f00c8c50 2024-05-08T23:40:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.997+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd3ec0065a0 con 0x7fd3f00c8c50 2024-05-08T23:40:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.998+0000 7fd3e67fc700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 0x7fd3d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.998+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd3ec088560 con 0x7fd3f00c8c50 2024-05-08T23:40:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:11.999+0000 7fd3f5ace700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 0x7fd3d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.000+0000 7fd3f5ace700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 0x7fd3d406d150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8003d90 tx=0x7fd3e80091f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:12.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.008+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d80052f0 con 0x7fd3f00c8c50 2024-05-08T23:40:12.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.024+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd3ec017080 con 0x7fd3f00c8c50 2024-05-08T23:40:12.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.937+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd3d8005b20 con 0x7fd3f00c8c50 2024-05-08T23:40:12.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.952+0000 7fd3e67fc700 1 -- 172.21.15.33:0/4249245849 <== mon.2 v2:172.21.15.42:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fd3ec053700 con 0x7fd3f00c8c50 2024-05-08T23:40:12.954 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.961+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 msgr2=0x7fd3d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.961+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 0x7fd3d406d150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8003d90 tx=0x7fd3e80091f0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.961+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 msgr2=0x7fd3f0153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.961+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b450 tx=0x7fd3ec00b810 comp rx=0 tx=0).stop 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.961+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 shutdown_connections 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.962+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd3d406ac90 0x7fd3d406d150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.962+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd3f00c8c50 0x7fd3f0153840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.962+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3f0070800 0x7fd3f014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.962+0000 7fd3f7d32700 1 --2- 172.21.15.33:0/4249245849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00c7330 0x7fd3f0153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:12.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.962+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 >> 172.21.15.33:0/4249245849 conn(0x7fd3f006a990 msgr2=0x7fd3f014fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:12.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.963+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 shutdown_connections 2024-05-08T23:40:12.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:12.963+0000 7fd3f7d32700 1 -- 172.21.15.33:0/4249245849 wait complete. 2024-05-08T23:40:16.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.717+0000 7fdeca609700 1 Processor -- start 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.717+0000 7fdeca609700 1 -- start start 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.718+0000 7fdeca609700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.718+0000 7fdeca609700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.718+0000 7fdeca609700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec406ba40 con 0x7fdec40c2770 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.718+0000 7fdeca609700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec415f180 con 0x7fdec40c7d80 2024-05-08T23:40:16.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.718+0000 7fdeca609700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c8210 con 0x7fdec41589d0 2024-05-08T23:40:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.719+0000 7fdec8ba6700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdec4070dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:16.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.719+0000 7fdec8ba6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.724+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.724+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:32812/0 (socket says 172.21.15.33:32812) 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.724+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 learned_addr learned my addr 172.21.15.33:0/330889895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.725+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.725+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.725+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:16.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.725+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:16.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.920+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdec4070dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:16.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.920+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:16.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.928+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:16.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.928+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:16.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.928+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:16.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.928+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:16.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:16.928+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:17.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.321+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdec4070dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:17.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.321+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.337+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.337+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.337+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.337+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:17.337+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:18.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.124+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdec4070dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:18.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.124+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.146+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.146+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.146+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.146+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:18.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:18.146+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec40c21a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec40c21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdec4070dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdec4070dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.42:6789/0 conn(0x7fdec40c2770 legacy=0x7fdec406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec4070930 con 0x7fdeb0003770 2024-05-08T23:40:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb000afb0 con 0x7fdeb0007420 2024-05-08T23:40:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.723+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.33:6789/0 conn(0x7fdeb0003770 legacy=0x7fdeb0005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:40:19.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.728+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.729+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.729+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.729+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.729+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:19.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.926+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.33:6789/0 conn(0x7fdeb0003770 legacy=0x7fdeb0005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:40:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.930+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.930+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.930+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.930+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:19.930+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:20.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.330+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.33:6789/0 conn(0x7fdeb0003770 legacy=0x7fdeb0005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:40:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.331+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:20.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.344+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:20.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.344+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:20.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.344+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:20.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:20.344+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.132+0000 7fdec3fff700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.33:6789/0 conn(0x7fdeb0003770 legacy=0x7fdeb0005bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.33:6789/0 2024-05-08T23:40:21.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.146+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.147+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.147+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:21.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.147+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:21.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:21.147+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:21.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:40:21.292 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:40:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 >> v1:172.21.15.33:6789/0 conn(0x7fdeb0003770 legacy=0x7fdeb0005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdeb0009fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdeb000a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb000afb0 con 0x7fdec40c7d80 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb000ab50 con 0x7fdeb0007420 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec1ffb700 1 -- 172.21.15.33:0/330889895 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb000acc0 con 0x7fdec41589d0 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec37fe700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdeb0009fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec37fe700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdeb0009fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.723+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 reap_dead start 2024-05-08T23:40:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.730+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdeb000a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.731+0000 7fdec3fff700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.737+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 msgr2=0x7fdeb00098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.737+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.737+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 msgr2=0x7fdeb0009fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.737+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdeb0009fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.737+0000 7fdec8ba6700 1 -- 172.21.15.33:0/330889895 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb000b180 con 0x7fdec41589d0 2024-05-08T23:40:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.738+0000 7fdec8ba6700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdeb000a560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdec4162d80 tx=0x7fdebc007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58fc8d07752cfc8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.744+0000 7fdec2ffd700 1 -- 172.21.15.33:0/330889895 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00e650 con 0x7fdec41589d0 2024-05-08T23:40:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.744+0000 7fdec2ffd700 1 -- 172.21.15.33:0/330889895 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc00e7b0 con 0x7fdec41589d0 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.745+0000 7fdec2ffd700 1 -- 172.21.15.33:0/330889895 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00eb60 con 0x7fdec41589d0 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 -- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdeb000a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdeb000a560 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdec4162d80 tx=0x7fdebc007f90 comp rx=0 tx=0).stop 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 -- 172.21.15.33:0/330889895 shutdown_connections 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.33:3300/0 conn(0x7fdec40c7d80 0x7fdeb0009fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 --2- 172.21.15.33:0/330889895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdeb000a560 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 --2- 172.21.15.33:0/330889895 >> v2:172.21.15.42:3300/0 conn(0x7fdeb0007420 0x7fdeb00098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.746+0000 7fdeca609700 1 -- 172.21.15.33:0/330889895 >> 172.21.15.33:0/330889895 conn(0x7fdec40c8360 msgr2=0x7fdec40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:22.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.750+0000 7fdeca609700 1 -- 172.21.15.33:0/330889895 shutdown_connections 2024-05-08T23:40:22.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.751+0000 7fdeca609700 1 -- 172.21.15.33:0/330889895 wait complete. 2024-05-08T23:40:22.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.755+0000 7fdeca609700 1 Processor -- start 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.756+0000 7fdeca609700 1 -- start start 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 0x7fdec40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec415b5e0 con 0x7fdeb0007420 2024-05-08T23:40:22.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec415ad20 con 0x7fdec40c7d80 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdeca609700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41493e0 con 0x7fdec41589d0 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdec3fff700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 msgr2=0x7fdec40c7300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 0x7fdec40c7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.757+0000 7fdec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:59526/0 (socket says 172.21.15.33:59526) 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 -- 172.21.15.33:0/1932062111 learned_addr learned my addr 172.21.15.33:0/1932062111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 -- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 msgr2=0x7fdec413d3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec413d3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 -- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 msgr2=0x7fdec40c7300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 0x7fdec40c7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 -- 172.21.15.33:0/1932062111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc007bf0 con 0x7fdec40c7d80 2024-05-08T23:40:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.758+0000 7fdec37fe700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80055d0 tx=0x7fdeb8001620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.763+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8001cf0 con 0x7fdec40c7d80 2024-05-08T23:40:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.763+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb8016660 con 0x7fdec40c7d80 2024-05-08T23:40:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.763+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80169d0 con 0x7fdec40c7d80 2024-05-08T23:40:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.763+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec4149660 con 0x7fdec40c7d80 2024-05-08T23:40:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.763+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec415af90 con 0x7fdec40c7d80 2024-05-08T23:40:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.765+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea40052f0 con 0x7fdec40c7d80 2024-05-08T23:40:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.777+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fdeb8016b30 con 0x7fdec40c7d80 2024-05-08T23:40:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.778+0000 7fdec0ff9700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 0x7fdeac06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.778+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fdeb808a280 con 0x7fdec40c7d80 2024-05-08T23:40:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.778+0000 7fdec3fff700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 0x7fdeac06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.778+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fdeb808a700 con 0x7fdec40c7d80 2024-05-08T23:40:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:22.779+0000 7fdec3fff700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 0x7fdeac06d280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009180 tx=0x7fdeb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.581+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdea4005870 con 0x7fdec40c7d80 2024-05-08T23:40:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.591+0000 7fdec0ff9700 1 -- 172.21.15.33:0/1932062111 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fdeb8055370 con 0x7fdec40c7d80 2024-05-08T23:40:23.593 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.600+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 msgr2=0x7fdeac06d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.600+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 0x7fdeac06d280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009180 tx=0x7fdeb4008040 comp rx=0 tx=0).stop 2024-05-08T23:40:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.600+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 msgr2=0x7fdec4159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.600+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80055d0 tx=0x7fdeb8001620 comp rx=0 tx=0).stop 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 shutdown_connections 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdeac06adc0 0x7fdeac06d280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fdec41589d0 0x7fdec413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdeb0007420 0x7fdec40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 --2- 172.21.15.33:0/1932062111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec40c7d80 0x7fdec4159080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.605+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 >> 172.21.15.33:0/1932062111 conn(0x7fdec40c8360 msgr2=0x7fdec414a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:23.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.606+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 shutdown_connections 2024-05-08T23:40:23.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.607+0000 7fdeca609700 1 -- 172.21.15.33:0/1932062111 wait complete. 2024-05-08T23:40:23.633 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-08T23:40:23.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-08T23:40:23.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:40:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.925+0000 7efd63120700 1 Processor -- start 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.925+0000 7efd63120700 1 -- start start 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 0x7efd5c0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7efd5c0c70c0 0x7efd5c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0c8110 con 0x7efd5c150510 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0c7e30 con 0x7efd5c0c70c0 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd63120700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0c6300 con 0x7efd5c156db0 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd60ebc700 1 -- >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 msgr2=0x7efd5c0c7780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.927+0000 7efd60ebc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 0x7efd5c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.935+0000 7efd616bd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.935+0000 7efd616bd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:32860/0 (socket says 172.21.15.33:32860) 2024-05-08T23:40:23.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.935+0000 7efd616bd700 1 -- 172.21.15.33:0/4203094370 learned_addr learned my addr 172.21.15.33:0/4203094370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 -- 172.21.15.33:0/4203094370 >> v2:172.21.15.42:3300/0 conn(0x7efd5c0c70c0 msgr2=0x7efd5c1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 --2- 172.21.15.33:0/4203094370 >> v2:172.21.15.42:3300/0 conn(0x7efd5c0c70c0 0x7efd5c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 -- 172.21.15.33:0/4203094370 >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 msgr2=0x7efd5c0c7780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:23.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 --2- 172.21.15.33:0/4203094370 >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 0x7efd5c0c7780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 -- 172.21.15.33:0/4203094370 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0c65e0 con 0x7efd5c156db0 2024-05-08T23:40:23.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.940+0000 7efd616bd700 1 --2- 172.21.15.33:0/4203094370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06bbe0 tx=0x7efd5400d650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abc7173c8889b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.941+0000 7efd5b7fe700 1 -- 172.21.15.33:0/4203094370 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54007d30 con 0x7efd5c156db0 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.941+0000 7efd5b7fe700 1 -- 172.21.15.33:0/4203094370 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54007e90 con 0x7efd5c156db0 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.941+0000 7efd5b7fe700 1 -- 172.21.15.33:0/4203094370 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54012710 con 0x7efd5c156db0 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 -- 172.21.15.33:0/4203094370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 msgr2=0x7efd5c06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 --2- 172.21.15.33:0/4203094370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efd5c06bbe0 tx=0x7efd5400d650 comp rx=0 tx=0).stop 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 -- 172.21.15.33:0/4203094370 shutdown_connections 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 --2- 172.21.15.33:0/4203094370 >> v2:172.21.15.33:3300/0 conn(0x7efd5c150510 0x7efd5c0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 --2- 172.21.15.33:0/4203094370 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c156db0 0x7efd5c06a990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 --2- 172.21.15.33:0/4203094370 >> v2:172.21.15.42:3300/0 conn(0x7efd5c0c70c0 0x7efd5c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.942+0000 7efd63120700 1 -- 172.21.15.33:0/4203094370 >> 172.21.15.33:0/4203094370 conn(0x7efd5c0c8360 msgr2=0x7efd5c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:23.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.943+0000 7efd63120700 1 -- 172.21.15.33:0/4203094370 shutdown_connections 2024-05-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.943+0000 7efd63120700 1 -- 172.21.15.33:0/4203094370 wait complete. 2024-05-08T23:40:23.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.945+0000 7efd63120700 1 Processor -- start 2024-05-08T23:40:23.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.946+0000 7efd63120700 1 -- start start 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 0x7efd5c149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd5c156db0 0x7efd5c0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c13f600 0x7efd5c14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c070e40 con 0x7efd5c156db0 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c071fb0 con 0x7efd5c13f600 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd63120700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c071d10 con 0x7efd5c150510 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd5bfff700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd5c156db0 msgr2=0x7efd5c0715b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:23.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.947+0000 7efd5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd5c156db0 0x7efd5c0715b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 0x7efd5c149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 0x7efd5c149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:32870/0 (socket says 172.21.15.33:32870) 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 -- 172.21.15.33:0/3468481646 learned_addr learned my addr 172.21.15.33:0/3468481646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c13f600 msgr2=0x7efd5c14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c13f600 0x7efd5c14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd5c156db0 msgr2=0x7efd5c0715b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd5c156db0 0x7efd5c0715b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:23.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.948+0000 7efd60ebc700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5400d050 con 0x7efd5c150510 2024-05-08T23:40:23.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd60ebc700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 0x7efd5c149250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd5000b4d0 tx=0x7efd5000b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50003e50 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50005070 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50005420 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c148b10 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.949+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c071090 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.951+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7efd50005580 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.952+0000 7efd63120700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 0x7efd5c070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.952+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c074a20 con 0x7efd5c14f510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.952+0000 7efd597fa700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7efd4006ac70 0x7efd4006d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.952+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7efd500887a0 con 0x7efd5c150510 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.952+0000 7efd616bd700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7efd4006ac70 0x7efd4006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.953+0000 7efd5bfff700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 0x7efd5c070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.953+0000 7efd616bd700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7efd4006ac70 0x7efd4006d130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd54007210 tx=0x7efd5400a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.953+0000 7efd5bfff700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd5c157240 con 0x7efd5c14f510 2024-05-08T23:40:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.953+0000 7efd5bfff700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 0x7efd5c070010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0c2270 tx=0x7efd4800c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48005bf0 con 0x7efd5c14f510 2024-05-08T23:40:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efd48005d50 con 0x7efd5c14f510 2024-05-08T23:40:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 msgr2=0x7efd5c070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 0x7efd5c070010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0c2270 tx=0x7efd4800c710 comp rx=0 tx=0).stop 2024-05-08T23:40:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 msgr2=0x7efd5c070010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.959+0000 7efd597fa700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c14f510 0x7efd5c070010 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0c2270 tx=0x7efd4800c710 comp rx=0 tx=0).stop 2024-05-08T23:40:23.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.979+0000 7efd63120700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 0x7efd5c147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:23.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.979+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c070960 con 0x7efd5c070550 2024-05-08T23:40:23.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.980+0000 7efd5bfff700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 0x7efd5c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.981+0000 7efd5bfff700 1 -- 172.21.15.33:0/3468481646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efd5c0c6ba0 con 0x7efd5c070550 2024-05-08T23:40:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.983+0000 7efd5bfff700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 0x7efd5c147460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd48011000 tx=0x7efd48004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4801f070 con 0x7efd5c070550 2024-05-08T23:40:23.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4931 (secure 0 0 0) 0x7efd480076f0 con 0x7efd5c070550 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 msgr2=0x7efd5c147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 0x7efd5c147460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd48011000 tx=0x7efd48004750 comp rx=0 tx=0).stop 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 msgr2=0x7efd5c147460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.984+0000 7efd597fa700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd5c070550 0x7efd5c147460 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd48011000 tx=0x7efd48004750 comp rx=0 tx=0).stop 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 66, 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:23.993 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.985+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7efd4006ac70 msgr2=0x7efd4006d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.985+0000 7efd63120700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7efd4006ac70 0x7efd4006d130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd54007210 tx=0x7efd5400a000 comp rx=0 tx=0).stop 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.985+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 msgr2=0x7efd5c149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.985+0000 7efd63120700 1 --2- 172.21.15.33:0/3468481646 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7efd5c150510 0x7efd5c149250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd5000b4d0 tx=0x7efd5000b890 comp rx=0 tx=0).stop 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.989+0000 7efd616bd700 1 -- 172.21.15.33:0/3468481646 reap_dead start 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.989+0000 7efd616bd700 1 -- 172.21.15.33:0/3468481646 reap_dead start 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.994+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 shutdown_connections 2024-05-08T23:40:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.994+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 >> 172.21.15.33:0/3468481646 conn(0x7efd5c0c8360 msgr2=0x7efd5c0c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.998+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 shutdown_connections 2024-05-08T23:40:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:23.999+0000 7efd63120700 1 -- 172.21.15.33:0/3468481646 wait complete. 2024-05-08T23:40:24.027 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:40:24.027 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-08T23:40:24.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.342+0000 7f27d465f700 1 Processor -- start 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.344+0000 7f27d465f700 1 -- start start 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.346+0000 7f27d465f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 0x7f27cc070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.346+0000 7f27d465f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f27cc0713d0 0x7f27cc070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.346+0000 7f27d465f700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc072e40 con 0x7f27cc0c9050 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.346+0000 7f27d465f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc06b920 con 0x7f27cc0713d0 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.346+0000 7f27d465f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc06aa30 con 0x7f27cc0c58c0 2024-05-08T23:40:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.352+0000 7f27d1bfa700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f27cc0c9050 0x7f27cc070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:38916/0 (socket says 172.21.15.33:38916) 2024-05-08T23:40:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.352+0000 7f27d1bfa700 1 -- 172.21.15.33:0/1839211236 learned_addr learned my addr 172.21.15.33:0/1839211236 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:40:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.354+0000 7f27d23fb700 1 --2- 172.21.15.33:0/1839211236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 0x7f27cc070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.355+0000 7f27d2bfc700 1 --2- 172.21.15.33:0/1839211236 >> v2:172.21.15.42:3300/0 conn(0x7f27cc0713d0 0x7f27cc070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.357+0000 7f27d23fb700 1 -- 172.21.15.33:0/1839211236 >> v2:172.21.15.42:3300/0 conn(0x7f27cc0713d0 msgr2=0x7f27cc070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.357+0000 7f27d23fb700 1 --2- 172.21.15.33:0/1839211236 >> v2:172.21.15.42:3300/0 conn(0x7f27cc0713d0 0x7f27cc070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.357+0000 7f27d23fb700 1 -- 172.21.15.33:0/1839211236 >> v1:172.21.15.42:6789/0 conn(0x7f27cc0c9050 legacy=0x7f27cc070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.357+0000 7f27d23fb700 1 -- 172.21.15.33:0/1839211236 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc06ad10 con 0x7f27cc0c58c0 2024-05-08T23:40:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.357+0000 7f27d23fb700 1 --2- 172.21.15.33:0/1839211236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 0x7f27cc070010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009840 tx=0x7f27bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c4643f00115cece server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.358+0000 7f27d13f9700 1 -- 172.21.15.33:0/1839211236 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc004230 con 0x7f27cc0c58c0 2024-05-08T23:40:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.358+0000 7f27d13f9700 1 -- 172.21.15.33:0/1839211236 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc004390 con 0x7f27cc0c58c0 2024-05-08T23:40:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.358+0000 7f27d13f9700 1 -- 172.21.15.33:0/1839211236 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0036a0 con 0x7f27cc0c58c0 2024-05-08T23:40:24.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.359+0000 7f27d465f700 1 -- 172.21.15.33:0/1839211236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 msgr2=0x7f27cc070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:24.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.359+0000 7f27d465f700 1 --2- 172.21.15.33:0/1839211236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 0x7f27cc070010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009840 tx=0x7f27bc009b50 comp rx=0 tx=0).stop 2024-05-08T23:40:24.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.369+0000 7f27d465f700 1 -- 172.21.15.33:0/1839211236 shutdown_connections 2024-05-08T23:40:24.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.370+0000 7f27d465f700 1 --2- 172.21.15.33:0/1839211236 >> v2:172.21.15.42:3300/0 conn(0x7f27cc0713d0 0x7f27cc070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.370+0000 7f27d465f700 1 --2- 172.21.15.33:0/1839211236 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c58c0 0x7f27cc070010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.370+0000 7f27d465f700 1 -- 172.21.15.33:0/1839211236 >> 172.21.15.33:0/1839211236 conn(0x7f27cc0c8360 msgr2=0x7f27cc0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:24.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.371+0000 7f27d465f700 1 -- 172.21.15.33:0/1839211236 shutdown_connections 2024-05-08T23:40:24.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.371+0000 7f27d465f700 1 -- 172.21.15.33:0/1839211236 wait complete. 2024-05-08T23:40:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.373+0000 7f27d465f700 1 Processor -- start 2024-05-08T23:40:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.373+0000 7f27d465f700 1 -- start start 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27cc0713d0 0x7f27cc1fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27cc0c58c0 0x7f27cc20e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 0x7f27cc08c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc20e8b0 con 0x7f27cc0713d0 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1f9440 con 0x7f27cc0c58c0 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d465f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc168c30 con 0x7f27cc0c9050 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d23fb700 1 -- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27cc0713d0 msgr2=0x7f27cc1fc720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:3300/0 2024-05-08T23:40:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.374+0000 7f27d23fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27cc0713d0 0x7f27cc1fc720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 0x7f27cc08c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 0x7f27cc08c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:32876/0 (socket says 172.21.15.33:32876) 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 learned_addr learned my addr 172.21.15.33:0/3880288150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27cc0c58c0 msgr2=0x7f27cc20e370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27cc0c58c0 0x7f27cc20e370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27cc0713d0 msgr2=0x7f27cc1fc720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27cc0713d0 0x7f27cc1fc720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.375+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc0094a0 con 0x7f27cc0c9050 2024-05-08T23:40:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.378+0000 7f27d2bfc700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 0x7f27cc08c300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27c400bd90 tx=0x7f27c40085d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.380+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4010040 con 0x7f27cc0c9050 2024-05-08T23:40:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.380+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4007e30 con 0x7f27cc0c9050 2024-05-08T23:40:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.380+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4014660 con 0x7f27cc0c9050 2024-05-08T23:40:24.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.380+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc168960 con 0x7f27cc0c9050 2024-05-08T23:40:24.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.380+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc168ef0 con 0x7f27cc0c9050 2024-05-08T23:40:24.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.383+0000 7f27d465f700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 0x7f27cc20f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.383+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc074a20 con 0x7f27cc20ed60 2024-05-08T23:40:24.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.384+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f27c4008040 con 0x7f27cc0c9050 2024-05-08T23:40:24.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.384+0000 7f27c2ffd700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f27b006ac90 0x7f27b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.385+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f27c4089630 con 0x7f27cc0c9050 2024-05-08T23:40:24.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.385+0000 7f27d1bfa700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f27b006ac90 0x7f27b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.386+0000 7f27d1bfa700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f27b006ac90 0x7f27b006d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27cc16b640 tx=0x7f27c8006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:24.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.388+0000 7f27d23fb700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 0x7f27cc20f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.389+0000 7f27d23fb700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27cc167e40 con 0x7f27cc20ed60 2024-05-08T23:40:24.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.389+0000 7f27d23fb700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 0x7f27cc20f140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009840 tx=0x7f27bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc003b30 con 0x7f27cc20ed60 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f27bc01f070 con 0x7f27cc20ed60 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 msgr2=0x7f27cc20f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 0x7f27cc20f140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009840 tx=0x7f27bc004750 comp rx=0 tx=0).stop 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 msgr2=0x7f27cc20f140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.392+0000 7f27c2ffd700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc20ed60 0x7f27cc20f140 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009840 tx=0x7f27bc004750 comp rx=0 tx=0).stop 2024-05-08T23:40:24.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.431+0000 7f27d465f700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 0x7f27cc1f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:24.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.431+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1f9050 con 0x7f27cc1f8700 2024-05-08T23:40:24.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.435+0000 7f27d23fb700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 0x7f27cc1f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:24.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.436+0000 7f27d23fb700 1 -- 172.21.15.33:0/3880288150 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f27cc0cb100 con 0x7f27cc1f8700 2024-05-08T23:40:24.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.453+0000 7f27d23fb700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 0x7f27cc1f8b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27cc06cc60 tx=0x7f27bc009210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc02a620 con 0x7f27cc1f8700 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4956 (secure 0 0 0) 0x7f27bc02a780 con 0x7f27cc1f8700 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 msgr2=0x7f27cc1f8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 0x7f27cc1f8b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27cc06cc60 tx=0x7f27bc009210 comp rx=0 tx=0).stop 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 msgr2=0x7f27cc1f8b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:24.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.463+0000 7f27c2ffd700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc1f8700 0x7f27cc1f8b10 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27cc06cc60 tx=0x7f27bc009210 comp rx=0 tx=0).stop 2024-05-08T23:40:24.467 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 66, 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 2, 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:40:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:24.480 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:24.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f27b006ac90 msgr2=0x7f27b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f27b006ac90 0x7f27b006d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f27cc16b640 tx=0x7f27c8006bf0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 msgr2=0x7f27cc08c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 --2- 172.21.15.33:0/3880288150 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f27cc0c9050 0x7f27cc08c300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27c400bd90 tx=0x7f27c40085d0 comp rx=0 tx=0).stop 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 reap_dead start 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d2bfc700 1 -- 172.21.15.33:0/3880288150 reap_dead start 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 shutdown_connections 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 >> 172.21.15.33:0/3880288150 conn(0x7f27cc0c8360 msgr2=0x7f27cc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.471+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 shutdown_connections 2024-05-08T23:40:24.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:24.472+0000 7f27d465f700 1 -- 172.21.15.33:0/3880288150 wait complete. 2024-05-08T23:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-08T23:40:24.516 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:Syncing()...1 sec 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:FFSB benchmark finished at: Wed May 8 23:40:39 2024 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:Results: 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:Benchmark took 335.62 sec 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:Total Results 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout:=============== 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout: ======= ============ ========= ======= =========== ========== 2024-05-08T23:40:39.679 INFO:tasks.workunit.client.0.smithi033.stdout: write : 10848000 32322.24 100.000% 100.000% 126MB/sec 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout:- 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout:32322.24 Transactions per Second 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout:Throughput Results 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout:=================== 2024-05-08T23:40:39.680 INFO:tasks.workunit.client.0.smithi033.stdout:Write Throughput: 126MB/sec 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout:System Call Latency statistics in millisecs 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout:===== 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: Min Avg Max Total Calls 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: ======== ======== ======== ============ 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout:[ open] 0.004000 0.054354 100.018997 8475 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-08T23:40:39.731 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[7] 20.000000 - 50.000000 : 3 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[9] 5.000000 - 10.000000 : 2 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[10] 2.000000 - 5.000000 : 3 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[11] 1.000000 - 2.000000 : 3 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[12] 0.500000 - 1.000000 : 2 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[13] 0.200000 - 0.500000 : 16 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[14] 0.100000 - 0.200000 : 12 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[15] 0.050000 - 0.100000 : 23 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[16] 0.020000 - 0.050000 : 1927 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[17] 0.010000 - 0.020000 : 4975 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[18] 0.000000 - 0.010000 : 1507 2024-05-08T23:40:39.732 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout:[ write] 0.001000 0.937205 29335.250000 10848000 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[0] 5000.000000 - 10000.000000 : 357 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[1] 2000.000000 - 5000.000000 : 846 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[2] 1000.000000 - 2000.000000 : 600 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[3] 500.000000 - 1000.000000 : 522 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[4] 200.000000 - 500.000000 : 646 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[5] 100.000000 - 200.000000 : 541 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[6] 50.000000 - 100.000000 : 685 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[7] 20.000000 - 50.000000 : 1015 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[8] 10.000000 - 20.000000 : 697 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[9] 5.000000 - 10.000000 : 546 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[10] 2.000000 - 5.000000 : 437 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[11] 1.000000 - 2.000000 : 215 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[12] 0.500000 - 1.000000 : 257 2024-05-08T23:40:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[13] 0.200000 - 0.500000 : 519 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[14] 0.100000 - 0.200000 : 542 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[15] 0.050000 - 0.100000 : 1989 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[16] 0.020000 - 0.050000 : 8851 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[17] 0.010000 - 0.020000 : 100731 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[18] 0.000000 - 0.010000 : 10727789 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout:[ lseek] 0.000000 0.001012 142.011993 10848000 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[5] 100.000000 - 200.000000 : 5 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[6] 50.000000 - 100.000000 : 12 2024-05-08T23:40:39.734 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[7] 20.000000 - 50.000000 : 41 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[8] 10.000000 - 20.000000 : 49 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[9] 5.000000 - 10.000000 : 25 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[10] 2.000000 - 5.000000 : 28 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[11] 1.000000 - 2.000000 : 28 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[12] 0.500000 - 1.000000 : 48 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[13] 0.200000 - 0.500000 : 65 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[14] 0.100000 - 0.200000 : 87 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[15] 0.050000 - 0.100000 : 235 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[16] 0.020000 - 0.050000 : 1147 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[17] 0.010000 - 0.020000 : 1324 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[18] 0.000000 - 0.010000 : 10844906 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout:[ close] 0.001000 0.004523 0.054000 8475 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-05-08T23:40:39.735 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-05-08T23:40:39.736 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[12] 0.500000 - 1.000000 : 0 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[15] 0.050000 - 0.100000 : 1 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[16] 0.020000 - 0.050000 : 11 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[17] 0.010000 - 0.020000 : 30 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: msec_range[18] 0.000000 - 0.010000 : 8433 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout:3.2% User Time 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout:11.6% System Time 2024-05-08T23:40:39.737 INFO:tasks.workunit.client.0.smithi033.stdout:14.7% CPU Utilization 2024-05-08T23:40:43.014 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T23:40:43.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T23:40:43.051 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-05-08T23:40:43.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T23:40:43.606 DEBUG:teuthology.parallel:result is None 2024-05-08T23:40:43.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T23:40:43.695 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T23:40:43.695 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-05-08T23:40:43.705 INFO:tasks.mon_thrash:joining mon_thrasher 2024-05-08T23:40:44.517 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T23:40:44.519 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-08T23:40:44.519 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T23:40:44.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T23:40:44.529 INFO:tasks.ceph.mon.a:Started 2024-05-08T23:40:44.529 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:40:44.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.763+0000 7f920a359700 1 Processor -- start 2024-05-08T23:40:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.764+0000 7f920a359700 1 -- start start 2024-05-08T23:40:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f920a359700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 0x7f92040c6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f920a359700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 0x7f920406b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f920a359700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920406b5e0 con 0x7f92040c9050 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f920a359700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9204072720 con 0x7f92040c67a0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f920a359700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920415a2e0 con 0x7f92041545c0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f92088f6700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 0x7f920406b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f9203fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 0x7f92040c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f92088f6700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 0x7f920406b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:48158/0 (socket says 172.21.15.33:48158) 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.765+0000 7f92088f6700 1 -- 172.21.15.33:0/1694258826 learned_addr learned my addr 172.21.15.33:0/1694258826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.766+0000 7f9203fff700 1 -- 172.21.15.33:0/1694258826 >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 msgr2=0x7f920406b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.766+0000 7f9203fff700 1 --2- 172.21.15.33:0/1694258826 >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 0x7f920406b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.766+0000 7f9203fff700 1 -- 172.21.15.33:0/1694258826 >> v1:172.21.15.42:6789/0 conn(0x7f92040c9050 legacy=0x7f920406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.766+0000 7f9203fff700 1 -- 172.21.15.33:0/1694258826 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920415a580 con 0x7f92041545c0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.767+0000 7f9203fff700 1 --2- 172.21.15.33:0/1694258826 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 0x7f92040c6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009840 tx=0x7f91f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9aec189b990ad9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.768+0000 7f9202ffd700 1 -- 172.21.15.33:0/1694258826 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4004230 con 0x7f92041545c0 2024-05-08T23:40:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.768+0000 7f9202ffd700 1 -- 172.21.15.33:0/1694258826 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4004390 con 0x7f92041545c0 2024-05-08T23:40:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.768+0000 7f9202ffd700 1 -- 172.21.15.33:0/1694258826 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40036a0 con 0x7f92041545c0 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 -- 172.21.15.33:0/1694258826 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 msgr2=0x7f92040c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 --2- 172.21.15.33:0/1694258826 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 0x7f92040c6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009840 tx=0x7f91f4009b50 comp rx=0 tx=0).stop 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 -- 172.21.15.33:0/1694258826 shutdown_connections 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 --2- 172.21.15.33:0/1694258826 >> v2:172.21.15.42:3300/0 conn(0x7f92040c67a0 0x7f920406b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 --2- 172.21.15.33:0/1694258826 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92041545c0 0x7f92040c6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 -- 172.21.15.33:0/1694258826 >> 172.21.15.33:0/1694258826 conn(0x7f92040c8360 msgr2=0x7f92040c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 -- 172.21.15.33:0/1694258826 shutdown_connections 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.769+0000 7f920a359700 1 -- 172.21.15.33:0/1694258826 wait complete. 2024-05-08T23:40:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.771+0000 7f920a359700 1 Processor -- start 2024-05-08T23:40:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.773+0000 7f920a359700 1 -- start start 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92040c67a0 0x7f920415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92040c9050 0x7f9204201000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041fdc40 con 0x7f92040c9050 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041fd0d0 con 0x7f92041545c0 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f920a359700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92041fe060 con 0x7f92040c67a0 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f92088f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f92088f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:48172/0 (socket says 172.21.15.33:48172) 2024-05-08T23:40:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.774+0000 7f92088f6700 1 -- 172.21.15.33:0/646523237 learned_addr learned my addr 172.21.15.33:0/646523237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 -- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92040c67a0 msgr2=0x7f920415d430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92040c67a0 0x7f920415d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 -- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92040c9050 msgr2=0x7f9204201000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92040c9050 0x7f9204201000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 -- 172.21.15.33:0/646523237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40094a0 con 0x7f92041545c0 2024-05-08T23:40:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f92088f6700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91fc007b00 tx=0x7f91fc007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc003c20 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00f040 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00d9b0 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.775+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204201540 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.776+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920415db90 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.777+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92041fdd80 con 0x7f92041545c0 2024-05-08T23:40:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.778+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f91fc00db10 con 0x7f92041545c0 2024-05-08T23:40:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.778+0000 7f9200ff9700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 0x7f91f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.778+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f91fc0891a0 con 0x7f92041545c0 2024-05-08T23:40:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.778+0000 7f9203fff700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 0x7f91f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.779+0000 7f9203fff700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 0x7f91f006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009b90 tx=0x7f91f4013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:44.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:44.787+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f92041fdd80 con 0x7f92041545c0 2024-05-08T23:40:45.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.330+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f92041fdd80 con 0x7f92041545c0 2024-05-08T23:40:45.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.335+0000 7f9200ff9700 1 -- 172.21.15.33:0/646523237 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f92041fdd80 con 0x7f92041545c0 2024-05-08T23:40:45.333 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 msgr2=0x7f91f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 0x7f91f006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009b90 tx=0x7f91f4013040 comp rx=0 tx=0).stop 2024-05-08T23:40:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 msgr2=0x7f920415b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:45.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91fc007b00 tx=0x7f91fc007ec0 comp rx=0 tx=0).stop 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 shutdown_connections 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f91f006ac90 0x7f91f006d150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92041545c0 0x7f920415b710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f92040c67a0 0x7f920415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 --2- 172.21.15.33:0/646523237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92040c9050 0x7f9204201000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:45.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.339+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 >> 172.21.15.33:0/646523237 conn(0x7f92040c8360 msgr2=0x7f920415a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:45.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.340+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 shutdown_connections 2024-05-08T23:40:45.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:45.340+0000 7f920a359700 1 -- 172.21.15.33:0/646523237 wait complete. 2024-05-08T23:40:48.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:48.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.505+0000 7fe00eb29700 1 Processor -- start 2024-05-08T23:40:48.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.506+0000 7fe00eb29700 1 -- start start 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.506+0000 7fe00eb29700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe00815b220 0x7fe008157b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.506+0000 7fe00eb29700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00815ad90 con 0x7fe00806bd10 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.506+0000 7fe00eb29700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe008071780 con 0x7fe0080712f0 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.506+0000 7fe00eb29700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00806c800 con 0x7fe00815b220 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe00c8c5700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe00806bd10 0x7fe00806c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:38834/0 (socket says 172.21.15.33:38834) 2024-05-08T23:40:48.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe00c8c5700 1 -- 172.21.15.33:0/4063349106 learned_addr learned my addr 172.21.15.33:0/4063349106 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe007fff700 1 --2- 172.21.15.33:0/4063349106 >> v2:172.21.15.42:3300/0 conn(0x7fe00815b220 0x7fe008157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe007fff700 1 -- 172.21.15.33:0/4063349106 >> v1:172.21.15.42:6789/0 conn(0x7fe0080712f0 legacy=0x7fe008150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe007fff700 1 -- 172.21.15.33:0/4063349106 >> v1:172.21.15.33:6789/0 conn(0x7fe00806bd10 legacy=0x7fe00806c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe007fff700 1 -- 172.21.15.33:0/4063349106 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080b2fe0 con 0x7fe00815b220 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.507+0000 7fe007fff700 1 --2- 172.21.15.33:0/4063349106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe008157b60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28f1283f3d7fc91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe0077fe700 1 -- 172.21.15.33:0/4063349106 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011830 con 0x7fe00815b220 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe0077fe700 1 -- 172.21.15.33:0/4063349106 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4011990 con 0x7fe00815b220 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe0077fe700 1 -- 172.21.15.33:0/4063349106 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011d20 con 0x7fe00815b220 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 -- 172.21.15.33:0/4063349106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 msgr2=0x7fe008157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 --2- 172.21.15.33:0/4063349106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe008157b60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4009b50 comp rx=0 tx=0).stop 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 -- 172.21.15.33:0/4063349106 shutdown_connections 2024-05-08T23:40:48.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 --2- 172.21.15.33:0/4063349106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe008157b60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 -- 172.21.15.33:0/4063349106 >> 172.21.15.33:0/4063349106 conn(0x7fe00806ce60 msgr2=0x7fe00806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 -- 172.21.15.33:0/4063349106 shutdown_connections 2024-05-08T23:40:48.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.508+0000 7fe00eb29700 1 -- 172.21.15.33:0/4063349106 wait complete. 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.510+0000 7fe00eb29700 1 Processor -- start 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.510+0000 7fe00eb29700 1 -- start start 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe0080712f0 0x7fe008149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe0081592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:48.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081469c0 con 0x7fe00806bd10 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081530c0 con 0x7fe00815b220 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00eb29700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe008147c30 con 0x7fe0080712f0 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47630/0 (socket says 172.21.15.33:47630) 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 -- 172.21.15.33:0/875333524 learned_addr learned my addr 172.21.15.33:0/875333524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe007fff700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe0080712f0 0x7fe008149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe0081592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 msgr2=0x7fe008153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 msgr2=0x7fe008153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:40:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00c8c5700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe0080712f0 msgr2=0x7fe008149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe0080712f0 0x7fe008149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 msgr2=0x7fe008153380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 -- 172.21.15.33:0/875333524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40094a0 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.511+0000 7fe00d0c6700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe0081592e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe00000cb90 tx=0x7fe00000cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.512+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000003c00 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.512+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000003d60 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.512+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0081461a0 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.512+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00000aaa0 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.512+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0081466e0 con 0x7fe00815b220 2024-05-08T23:40:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.513+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008073770 con 0x7fe00815b220 2024-05-08T23:40:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.513+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe00000ac00 con 0x7fe00815b220 2024-05-08T23:40:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.514+0000 7fe0057fa700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 0x7fdfec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.514+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fe000089430 con 0x7fe00815b220 2024-05-08T23:40:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.514+0000 7fe00c8c5700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 0x7fdfec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.514+0000 7fe00c8c5700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 0x7fdfec06d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007fd0 tx=0x7fdffc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:48.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.519+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe0000544a0 con 0x7fe00815b220 2024-05-08T23:40:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.849+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe008075460 con 0x7fe00815b220 2024-05-08T23:40:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.850+0000 7fe0057fa700 1 -- 172.21.15.33:0/875333524 <== mon.1 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fe000057ac0 con 0x7fe00815b220 2024-05-08T23:40:48.848 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:48.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 msgr2=0x7fdfec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 0x7fdfec06d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007fd0 tx=0x7fdffc009040 comp rx=0 tx=0).stop 2024-05-08T23:40:48.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 msgr2=0x7fe0081592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:48.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe0081592e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe00000cb90 tx=0x7fe00000cea0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 shutdown_connections 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fdfec06ac90 0x7fdfec06d150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe00815b220 0x7fe0081592e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00806bd10 0x7fe008153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 --2- 172.21.15.33:0/875333524 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe0080712f0 0x7fe008149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 >> 172.21.15.33:0/875333524 conn(0x7fe00806ce60 msgr2=0x7fe0080c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 shutdown_connections 2024-05-08T23:40:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:48.853+0000 7fe00eb29700 1 -- 172.21.15.33:0/875333524 wait complete. 2024-05-08T23:40:51.554 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:40:51.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:40:51.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:40:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.012+0000 7fc1baadb700 1 Processor -- start 2024-05-08T23:40:52.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.012+0000 7fc1baadb700 1 -- start start 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1baadb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1baadb700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1baadb700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b415ad90 con 0x7fc1b406c0f0 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1baadb700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b4157fe0 con 0x7fc1b4157b60 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1baadb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40b2fe0 con 0x7fc1b406bd10 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1b3fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1b8877700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1b3fff700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.33:60666/0 (socket says 172.21.15.33:60666) 2024-05-08T23:40:52.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 learned_addr learned my addr 172.21.15.33:0/481350365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.013+0000 7fc1b8877700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:55942/0 (socket says 172.21.15.33:55942) 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.014+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:52.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:52.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.215+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:52.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:52.616+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b8877700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:53.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:53.417+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 msgr2=0x7fc1b415b220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1b406bd10 0x7fc1b415b220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 msgr2=0x7fc1b40712f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.42:3300/0 conn(0x7fc1b4157b60 0x7fc1b40712f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 -- 172.21.15.33:0/481350365 >> v1:172.21.15.42:6789/0 conn(0x7fc1b406c0f0 legacy=0x7fc1b4150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 -- 172.21.15.33:0/481350365 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40b2a20 con 0x7fc194003750 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b27fc700 1 -- 172.21.15.33:0/481350365 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b4157fe0 con 0x7fc1940073e0 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.014+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.215+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.215+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.215+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:40:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.616+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.616+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:55.616+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:40:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:56.418+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:56.418+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T23:40:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:56.418+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T23:40:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:56.418+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:40:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:56.418+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:40:57.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.372+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37481387 0 0) 0x7fc1b40b2a20 con 0x7fc194003750 2024-05-08T23:40:57.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.372+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a00035b0 con 0x7fc194003750 2024-05-08T23:40:57.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302700580 0 0) 0x7fc1a00035b0 con 0x7fc194003750 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 msgr2=0x7fc1940098a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 --2- 172.21.15.33:0/481350365 >> v2:172.21.15.33:3300/0 conn(0x7fc1940073e0 0x7fc1940098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19400afa0 con 0x7fc194003750 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7fc1a40107a0 con 0x7fc194003750 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 >> v1:172.21.15.33:6789/0 conn(0x7fc194003750 legacy=0x7fc194005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b9078700 1 -- 172.21.15.33:0/481350365 reap_dead start 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 0x7fc1a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0005410 0x7fc1a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4157fe0 con 0x7fc1a0004850 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a000c2d0 con 0x7fc1a0003a50 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a000c490 con 0x7fc1a0005410 2024-05-08T23:40:57.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 0x7fc1a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b8877700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.373+0000 7fc1b9078700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0005410 0x7fc1a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0005410 msgr2=0x7fc1a000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0005410 0x7fc1a000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 msgr2=0x7fc1a0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1a0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 -- 172.21.15.33:0/481350365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00035b0 con 0x7fc1a0004850 2024-05-08T23:40:57.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.374+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 0x7fc1a0004cd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc19c019ae0 tx=0x7fc1a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a8a3e3d09e77b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.376+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c01e070 con 0x7fc1a0004850 2024-05-08T23:40:57.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.376+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00c040 con 0x7fc1a0004850 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.376+0000 7fc1b37fe700 1 -- 172.21.15.33:0/481350365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004240 con 0x7fc1a0004850 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 -- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 msgr2=0x7fc1a0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 0x7fc1a0004cd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc19c019ae0 tx=0x7fc1a0005880 comp rx=0 tx=0).stop 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 -- 172.21.15.33:0/481350365 shutdown_connections 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0005410 0x7fc1a000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0004850 0x7fc1a0004cd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 --2- 172.21.15.33:0/481350365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1a0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 -- 172.21.15.33:0/481350365 >> 172.21.15.33:0/481350365 conn(0x7fc1b406ce60 msgr2=0x7fc1b406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.377+0000 7fc1baadb700 1 -- 172.21.15.33:0/481350365 shutdown_connections 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.378+0000 7fc1baadb700 1 -- 172.21.15.33:0/481350365 wait complete. 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.379+0000 7fc1baadb700 1 Processor -- start 2024-05-08T23:40:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.379+0000 7fc1baadb700 1 -- start start 2024-05-08T23:40:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1b41dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 0x7fc1b41e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b41e0670 con 0x7fc1a0005410 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b414c020 con 0x7fc1a0003a50 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1baadb700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b414baa0 con 0x7fc1a0004850 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1b9078700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1b9078700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40026/0 (socket says 172.21.15.33:40026) 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.380+0000 7fc1b9078700 1 -- 172.21.15.33:0/2412292444 learned_addr learned my addr 172.21.15.33:0/2412292444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b8877700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1b41dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 0x7fc1b41e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 -- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 msgr2=0x7fc1b41e17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 0x7fc1b41e17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 -- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 msgr2=0x7fc1b41dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1b41dfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 -- 172.21.15.33:0/2412292444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4159740 con 0x7fc1a0005410 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b3fff700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 0x7fc1b41e17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:40:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b8877700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1b41dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b9078700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc194007350 tx=0x7fc1ac00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac001c30 con 0x7fc1a0005410 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac005de0 con 0x7fc1a0005410 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b414bd40 con 0x7fc1a0005410 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00d9f0 con 0x7fc1a0005410 2024-05-08T23:40:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.381+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b4159cc0 con 0x7fc1a0005410 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.382+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b4073770 con 0x7fc1a0005410 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.383+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc1ac00db50 con 0x7fc1a0005410 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.384+0000 7fc1b17fa700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 0x7fc19406d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.384+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fc1ac089460 con 0x7fc1a0005410 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.384+0000 7fc1b8877700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 0x7fc19406d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.384+0000 7fc1b8877700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 0x7fc19406d980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000c00 tx=0x7fc1a4017040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.389+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fc1ac054570 con 0x7fc1a0005410 2024-05-08T23:40:57.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.721+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc1b4159eb0 con 0x7fc1a0005410 2024-05-08T23:40:57.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.722+0000 7fc1b17fa700 1 -- 172.21.15.33:0/2412292444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fc1ac057b90 con 0x7fc1a0005410 2024-05-08T23:40:57.719 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 msgr2=0x7fc19406d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 0x7fc19406d980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4000c00 tx=0x7fc1a4017040 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 msgr2=0x7fc1b40cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc194007350 tx=0x7fc1ac00ff20 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 shutdown_connections 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc19406b4c0 0x7fc19406d980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a0005410 0x7fc1b40cbe90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1a0003a50 0x7fc1b41dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 --2- 172.21.15.33:0/2412292444 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc1a0004850 0x7fc1b41e17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 >> 172.21.15.33:0/2412292444 conn(0x7fc1b406ce60 msgr2=0x7fc1b414d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:57.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 shutdown_connections 2024-05-08T23:40:57.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.724+0000 7fc1baadb700 1 -- 172.21.15.33:0/2412292444 wait complete. 2024-05-08T23:40:57.736 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:40:57.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-08T23:40:57.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T23:40:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.877+0000 7f51ff4ea700 1 Processor -- start 2024-05-08T23:40:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.877+0000 7f51ff4ea700 1 -- start start 2024-05-08T23:40:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51ff4ea700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f51f80c9390 0x7f51f806d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51ff4ea700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f80b29b0 con 0x7f51f8158a50 2024-05-08T23:40:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51ff4ea700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f80c8f50 con 0x7f51f80c2890 2024-05-08T23:40:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51ff4ea700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f80c2d40 con 0x7f51f80c9390 2024-05-08T23:40:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51fd286700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f51f8158a50 0x7f51f80c77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56678/0 (socket says 172.21.15.33:56678) 2024-05-08T23:40:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.878+0000 7f51fd286700 1 -- 172.21.15.33:0/3554157738 learned_addr learned my addr 172.21.15.33:0/3554157738 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51fca85700 1 --2- 172.21.15.33:0/3554157738 >> v2:172.21.15.42:3300/0 conn(0x7f51f80c9390 0x7f51f806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889623542 0 0) 0x7f51f80b29b0 con 0x7f51f8158a50 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e40035b0 con 0x7f51f8158a50 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295101548 0 0) 0x7f51e40035b0 con 0x7f51f8158a50 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 >> v2:172.21.15.42:3300/0 conn(0x7f51f80c9390 msgr2=0x7f51f806d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 --2- 172.21.15.33:0/3554157738 >> v2:172.21.15.42:3300/0 conn(0x7f51f80c9390 0x7f51f806d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 >> v1:172.21.15.42:6789/0 conn(0x7f51f80c2890 legacy=0x7f51f8161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f80c8940 con 0x7f51f8158a50 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f51f4001670 con 0x7f51f8158a50 2024-05-08T23:40:57.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 >> v1:172.21.15.33:6789/0 conn(0x7f51f8158a50 legacy=0x7f51f80c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e4003a00 0x7f51e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4004850 0x7f51e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 0x7f51e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80c2d40 con 0x7f51e4005210 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e400c060 con 0x7f51e4003a00 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e400c220 con 0x7f51e4004850 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51fda87700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 0x7f51e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51fd286700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e4003a00 0x7f51e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.879+0000 7f51fca85700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4004850 0x7f51e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 -- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4004850 msgr2=0x7f51e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4004850 0x7f51e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 -- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e4003a00 msgr2=0x7f51e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e4003a00 0x7f51e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 -- 172.21.15.33:0/3554157738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40035b0 con 0x7f51e4005210 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 -- 172.21.15.33:0/3554157738 reap_dead start 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51fda87700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 0x7f51e400ba10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c290 tx=0x7f51e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6168f1c05c0ccda9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0001e70 con 0x7f51e4005210 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f000ce70 con 0x7f51e4005210 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51ebfff700 1 -- 172.21.15.33:0/3554157738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003c20 con 0x7f51e4005210 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 msgr2=0x7f51e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.880+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 0x7f51e400ba10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c290 tx=0x7f51e4005880 comp rx=0 tx=0).stop 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.881+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3554157738 shutdown_connections 2024-05-08T23:40:57.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.881+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3554157738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51e4005210 0x7f51e400ba10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.881+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3554157738 >> 172.21.15.33:0/3554157738 conn(0x7f51f806ce60 msgr2=0x7f51f806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:57.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.881+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3554157738 shutdown_connections 2024-05-08T23:40:57.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.881+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3554157738 wait complete. 2024-05-08T23:40:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.882+0000 7f51ff4ea700 1 Processor -- start 2024-05-08T23:40:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.882+0000 7f51ff4ea700 1 -- start start 2024-05-08T23:40:57.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4005210 0x7f51f814e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 0x7f51f814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f8153d40 0x7f51f81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8151610 con 0x7f51f814ed20 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f814f7d0 con 0x7f51f8153d40 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51ff4ea700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80c4750 con 0x7f51e4005210 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 0x7f51f814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 0x7f51f814f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40042/0 (socket says 172.21.15.33:40042) 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 -- 172.21.15.33:0/3144395453 learned_addr learned my addr 172.21.15.33:0/3144395453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fd286700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4005210 0x7f51f814e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fda87700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f8153d40 0x7f51f81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4005210 msgr2=0x7f51f814e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f51e4005210 0x7f51f814e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f8153d40 msgr2=0x7f51f81541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f8153d40 0x7f51f81541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8150c30 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.883+0000 7f51fca85700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 0x7f51f814f1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011970 tx=0x7f51ec011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.884+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec004240 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.884+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec0043a0 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.884+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f814e880 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.884+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0036a0 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.884+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8150300 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 0x7f51f8156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8074b70 con 0x7f51f8150520 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51fd286700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 0x7f51f8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51fd286700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51f80cb510 con 0x7f51f8150520 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f51ec003800 con 0x7f51f814ed20 2024-05-08T23:40:57.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.885+0000 7f51fd286700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 0x7f51f8156240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007fd0 tx=0x7f51f40075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f51dc06ac90 0x7f51dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f51ec089810 con 0x7f51f814ed20 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51fda87700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f51dc06ac90 0x7f51dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4007800 con 0x7f51f8150520 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f51f40128c0 con 0x7f51f8150520 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 msgr2=0x7f51f8156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 0x7f51f8156240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007fd0 tx=0x7f51f40075d0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 msgr2=0x7f51f8156240 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51e9ffb700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8150520 0x7f51f8156240 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.886+0000 7f51fda87700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f51dc06ac90 0x7f51dc06d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c030 tx=0x7f51f0005000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.894+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 0x7f51f81f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.894+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80c3830 con 0x7f51f8156780 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.894+0000 7f51fd286700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 0x7f51f81f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.894+0000 7f51fd286700 1 -- 172.21.15.33:0/3144395453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f51f80c8be0 con 0x7f51f8156780 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.894+0000 7f51fd286700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 0x7f51f81f7530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007fd0 tx=0x7f51f4001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:57.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4025070 con 0x7f51f8156780 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4459 (secure 0 0 0) 0x7f51f4015760 con 0x7f51f8156780 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 msgr2=0x7f51f81f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 0x7f51f81f7530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007fd0 tx=0x7f51f4001dc0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 msgr2=0x7f51f81f7530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51e9ffb700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f8156780 0x7f51f81f7530 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f51f4007fd0 tx=0x7f51f4001dc0 comp rx=0 tx=0).stop 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:57.893 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "leader", 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 74, 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:57.894 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:57.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:57.899 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:57.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:57.901 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:57.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:57.906 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:57.906 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f51dc06ac90 msgr2=0x7f51dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f51dc06ac90 0x7f51dc06d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c030 tx=0x7f51f0005000 comp rx=0 tx=0).stop 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 msgr2=0x7f51f814f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 --2- 172.21.15.33:0/3144395453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51f814ed20 0x7f51f814f1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51ec011970 tx=0x7f51ec011c80 comp rx=0 tx=0).stop 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51fda87700 1 -- 172.21.15.33:0/3144395453 reap_dead start 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51fda87700 1 -- 172.21.15.33:0/3144395453 reap_dead start 2024-05-08T23:40:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 shutdown_connections 2024-05-08T23:40:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 >> 172.21.15.33:0/3144395453 conn(0x7f51f806ce60 msgr2=0x7f51f81621d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 shutdown_connections 2024-05-08T23:40:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:57.895+0000 7f51ff4ea700 1 -- 172.21.15.33:0/3144395453 wait complete. 2024-05-08T23:40:57.908 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:57.908 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-08T23:40:57.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 Processor -- start 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 -- start start 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12280caad0 0x7f12280cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 0x7f122815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122815b350 con 0x7f1228154970 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12280cbff0 con 0x7f12280caad0 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.050+0000 7f122f0ea700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122815e8a0 con 0x7f12280cbbe0 2024-05-08T23:40:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12280caad0 0x7f12280cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f12280caad0 0x7f12280cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40074/0 (socket says 172.21.15.33:40074) 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f122ce86700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1228154970 0x7f122806ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56680/0 (socket says 172.21.15.33:56680) 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 -- 172.21.15.33:0/2706340409 learned_addr learned my addr 172.21.15.33:0/2706340409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f122d687700 1 --2- 172.21.15.33:0/2706340409 >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 0x7f122815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 -- 172.21.15.33:0/2706340409 >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 msgr2=0x7f122815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 --2- 172.21.15.33:0/2706340409 >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 0x7f122815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 -- 172.21.15.33:0/2706340409 >> v1:172.21.15.33:6789/0 conn(0x7f1228154970 legacy=0x7f122806ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 -- 172.21.15.33:0/2706340409 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122815a5d0 con 0x7f12280caad0 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f12277fe700 1 -- 172.21.15.33:0/2706340409 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683892669 0 0) 0x7f122815b350 con 0x7f1228154970 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f122d687700 1 --2- 172.21.15.33:0/2706340409 >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 0x7f122815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f1227fff700 1 --2- 172.21.15.33:0/2706340409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f12280cb6a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12280cd6b0 tx=0x7f1210009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6554b132a129c048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f12277fe700 1 -- 172.21.15.33:0/2706340409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000b870 con 0x7f12280caad0 2024-05-08T23:40:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f12277fe700 1 -- 172.21.15.33:0/2706340409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121000b9d0 con 0x7f12280caad0 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.051+0000 7f12277fe700 1 -- 172.21.15.33:0/2706340409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000bd60 con 0x7f12280caad0 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 -- 172.21.15.33:0/2706340409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 msgr2=0x7f12280cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 --2- 172.21.15.33:0/2706340409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f12280cb6a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12280cd6b0 tx=0x7f1210009bd0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 -- 172.21.15.33:0/2706340409 shutdown_connections 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 --2- 172.21.15.33:0/2706340409 >> v2:172.21.15.42:3300/0 conn(0x7f12280cbbe0 0x7f122815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 --2- 172.21.15.33:0/2706340409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f12280cb6a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 -- 172.21.15.33:0/2706340409 >> 172.21.15.33:0/2706340409 conn(0x7f122806cd80 msgr2=0x7f122806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 -- 172.21.15.33:0/2706340409 shutdown_connections 2024-05-08T23:40:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.052+0000 7f122f0ea700 1 -- 172.21.15.33:0/2706340409 wait complete. 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.053+0000 7f122f0ea700 1 Processor -- start 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.053+0000 7f122f0ea700 1 -- start start 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f122814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280cbbe0 0x7f122814e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1228154970 0x7f12281f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12280cb730 con 0x7f12280caad0 2024-05-08T23:40:58.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12281f0b40 con 0x7f12280cbbe0 2024-05-08T23:40:58.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122f0ea700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228154f50 con 0x7f1228154970 2024-05-08T23:40:58.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122ce86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f122814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122ce86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f122814dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40076/0 (socket says 172.21.15.33:40076) 2024-05-08T23:40:58.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.054+0000 7f122ce86700 1 -- 172.21.15.33:0/811326137 learned_addr learned my addr 172.21.15.33:0/811326137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f1227fff700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280cbbe0 0x7f122814e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122d687700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1228154970 0x7f12281f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1228154970 msgr2=0x7f12281f3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1228154970 0x7f12281f3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280cbbe0 msgr2=0x7f122814e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280cbbe0 0x7f122814e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009550 con 0x7f12280caad0 2024-05-08T23:40:58.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122ce86700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f122814dfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f121800c550 tx=0x7f121800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218004c80 con 0x7f12280caad0 2024-05-08T23:40:58.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218004e00 con 0x7f12280caad0 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218007310 con 0x7f12280caad0 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12281f0de0 con 0x7f12280caad0 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.055+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12281554f0 con 0x7f12280caad0 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.056+0000 7f122f0ea700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 0x7f12281f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.056+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228074690 con 0x7f12280c31c0 2024-05-08T23:40:58.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.056+0000 7f1227fff700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 0x7f12281f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f1227fff700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12280cb2b0 con 0x7f12280c31c0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1218007470 con 0x7f12280caad0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f1227fff700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 0x7f12281f3d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12280cd6b0 tx=0x7f1210004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f120806ac90 0x7f120806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1218089040 con 0x7f12280caad0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f122d687700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f120806ac90 0x7f120806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1210004450 con 0x7f12280c31c0 2024-05-08T23:40:58.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f12100045b0 con 0x7f12280c31c0 2024-05-08T23:40:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 msgr2=0x7f12281f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 0x7f12281f3d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12280cd6b0 tx=0x7f1210004750 comp rx=0 tx=0).stop 2024-05-08T23:40:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 msgr2=0x7f12281f3d40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.057+0000 7f12257fa700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12280c31c0 0x7f12281f3d40 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12280cd6b0 tx=0x7f1210004750 comp rx=0 tx=0).stop 2024-05-08T23:40:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.058+0000 7f122d687700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f120806ac90 0x7f120806d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1220009a80 tx=0x7f1220003a80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.065+0000 7f122f0ea700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 0x7f12281561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.065+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122815ab50 con 0x7f1228155d90 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.066+0000 7f1227fff700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 0x7f12281561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.066+0000 7f1227fff700 1 -- 172.21.15.33:0/811326137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f122815a870 con 0x7f1228155d90 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.066+0000 7f1227fff700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 0x7f12281561a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f12280cc1b0 tx=0x7f1210011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.066+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121002bb40 con 0x7f1228155d90 2024-05-08T23:40:58.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 <== mon.1 v2:172.21.15.42:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4940 (secure 0 0 0) 0x7f121002bca0 con 0x7f1228155d90 2024-05-08T23:40:58.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 msgr2=0x7f12281561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f12257fa700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 0x7f12281561a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f12280cc1b0 tx=0x7f1210011000 comp rx=0 tx=0).stop 2024-05-08T23:40:58.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f12257fa700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 msgr2=0x7f12281561a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:58.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f12257fa700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1228155d90 0x7f12281561a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.067 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:58.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:58.067 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 74, 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:58.068 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:58.069 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.070 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:58.071 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:58.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.074 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.077 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.078 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.079 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.080 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:58.081 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f120806ac90 msgr2=0x7f120806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f120806ac90 0x7f120806d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1220009a80 tx=0x7f1220003a80 comp rx=0 tx=0).stop 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 msgr2=0x7f122814dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 --2- 172.21.15.33:0/811326137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12280caad0 0x7f122814dfc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f121800c550 tx=0x7f121800c910 comp rx=0 tx=0).stop 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122d687700 1 -- 172.21.15.33:0/811326137 reap_dead start 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122d687700 1 -- 172.21.15.33:0/811326137 reap_dead start 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 shutdown_connections 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 >> 172.21.15.33:0/811326137 conn(0x7f122806cd80 msgr2=0x7f12281582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 shutdown_connections 2024-05-08T23:40:58.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.067+0000 7f122f0ea700 1 -- 172.21.15.33:0/811326137 wait complete. 2024-05-08T23:40:58.083 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:58.083 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-08T23:40:58.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T23:40:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.224+0000 7f58b94e9700 1 Processor -- start 2024-05-08T23:40:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.224+0000 7f58b94e9700 1 -- start start 2024-05-08T23:40:58.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b94e9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40c4c70 0x7f58b415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b94e9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58b40d00b0 0x7f58b40d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b94e9700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40c5c20 con 0x7f58b4154770 2024-05-08T23:40:58.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b94e9700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40b30b0 con 0x7f58b40d00b0 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b94e9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4154b50 con 0x7f58b40c4c70 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b37fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f58b40d00b0 0x7f58b40d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b2ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f58b4154770 0x7f58b40c60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56684/0 (socket says 172.21.15.33:56684) 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b2ffd700 1 -- 172.21.15.33:0/4002565780 learned_addr learned my addr 172.21.15.33:0/4002565780 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.225+0000 7f58b27fc700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40c4c70 0x7f58b415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b37fe700 1 -- 172.21.15.33:0/4002565780 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40c4c70 msgr2=0x7f58b415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b37fe700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40c4c70 0x7f58b415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b37fe700 1 -- 172.21.15.33:0/4002565780 >> v1:172.21.15.33:6789/0 conn(0x7f58b4154770 legacy=0x7f58b40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b37fe700 1 -- 172.21.15.33:0/4002565780 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40caed0 con 0x7f58b40d00b0 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b1ffb700 1 -- 172.21.15.33:0/4002565780 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116938777 0 0) 0x7f58b40c5c20 con 0x7f58b4154770 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b37fe700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b40d00b0 0x7f58b40d0540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009d90 tx=0x7f58ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a1adcf1cb0e884 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b1ffb700 1 -- 172.21.15.33:0/4002565780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00e070 con 0x7f58b40d00b0 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b1ffb700 1 -- 172.21.15.33:0/4002565780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00bbd0 con 0x7f58b40d00b0 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.226+0000 7f58b1ffb700 1 -- 172.21.15.33:0/4002565780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac012430 con 0x7f58b40d00b0 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 -- 172.21.15.33:0/4002565780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b40d00b0 msgr2=0x7f58b40d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b40d00b0 0x7f58b40d0540 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009d90 tx=0x7f58ac00b330 comp rx=0 tx=0).stop 2024-05-08T23:40:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 -- 172.21.15.33:0/4002565780 shutdown_connections 2024-05-08T23:40:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b40d00b0 0x7f58b40d0540 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 --2- 172.21.15.33:0/4002565780 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40c4c70 0x7f58b415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 -- 172.21.15.33:0/4002565780 >> 172.21.15.33:0/4002565780 conn(0x7f58b4066ae0 msgr2=0x7f58b4066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 -- 172.21.15.33:0/4002565780 shutdown_connections 2024-05-08T23:40:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.227+0000 7f58b94e9700 1 -- 172.21.15.33:0/4002565780 wait complete. 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.228+0000 7f58b94e9700 1 Processor -- start 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.228+0000 7f58b94e9700 1 -- start start 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58b40c4c70 0x7f58b4159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40d00b0 0x7f58b40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f3a20 con 0x7f58b4154770 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b41f3f50 con 0x7f58b40c4c70 2024-05-08T23:40:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b94e9700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4154de0 con 0x7f58b40d00b0 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40102/0 (socket says 172.21.15.33:40102) 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 -- 172.21.15.33:0/1104831435 learned_addr learned my addr 172.21.15.33:0/1104831435 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58b40c4c70 0x7f58b4159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b27fc700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40d00b0 0x7f58b40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40d00b0 msgr2=0x7f58b40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40d00b0 0x7f58b40cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58b40c4c70 msgr2=0x7f58b4159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58b40c4c70 0x7f58b4159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.229+0000 7f58b37fe700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009a70 con 0x7f58b4154770 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58b37fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f58b4073780 tx=0x7f58ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac012430 con 0x7f58b4154770 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac020070 con 0x7f58b4154770 2024-05-08T23:40:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b4155050 con 0x7f58b4154770 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004280 con 0x7f58b4154770 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.230+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40b57c0 con 0x7f58b4154770 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.231+0000 7f58b94e9700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 0x7f58b41f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.231+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4074690 con 0x7f58b40b5a30 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.231+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 0x7f58b41f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.231+0000 7f58b2ffd700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58b40c52f0 con 0x7f58b40b5a30 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.231+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 0x7f58b41f03c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f589c0014c0 tx=0x7f589c001880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f58ac0043e0 con 0x7f58b4154770 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f589406acc0 0x7f589406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f58ac022070 con 0x7f58b4154770 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58b27fc700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f589406acc0 0x7f589406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c017070 con 0x7f58b40b5a30 2024-05-08T23:40:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f589c012560 con 0x7f58b40b5a30 2024-05-08T23:40:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 msgr2=0x7f58b41f03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 0x7f58b41f03c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f589c0014c0 tx=0x7f589c001880 comp rx=0 tx=0).stop 2024-05-08T23:40:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 msgr2=0x7f58b41f03c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.232+0000 7f58ab7fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b40b5a30 0x7f58b41f03c0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f589c0014c0 tx=0x7f589c001880 comp rx=0 tx=0).stop 2024-05-08T23:40:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.233+0000 7f58b27fc700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f589406acc0 0x7f589406d180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58b414b320 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.240+0000 7f58b94e9700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 0x7f58b4147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.240+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40cabf0 con 0x7f58b4147330 2024-05-08T23:40:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.240+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 0x7f58b4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58b2ffd700 1 -- 172.21.15.33:0/1104831435 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f58b40c55d0 con 0x7f58b4147330 2024-05-08T23:40:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58b2ffd700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 0x7f58b4147740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f589c015040 tx=0x7f589c001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c022070 con 0x7f58b4147330 2024-05-08T23:40:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 <== mon.2 v2:172.21.15.42:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4967 (secure 0 0 0) 0x7f589c01dd00 con 0x7f58b4147330 2024-05-08T23:40:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 msgr2=0x7f58b4147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 0x7f58b4147740 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f589c015040 tx=0x7f589c001080 comp rx=0 tx=0).stop 2024-05-08T23:40:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 msgr2=0x7f58b4147740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T23:40:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.241+0000 7f58ab7fe700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f58b4147330 0x7f58b4147740 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "peon", 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "election_epoch": 74, 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum": [ 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: 0, 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: 1, 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: 2 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_age": 0, 2024-05-08T23:40:58.240 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "required_con": "2449958758054445060", 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "required_mon": [ 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.241 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_con": "4540138322906710015", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "quorum_mon": [ 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "outside_quorum": [], 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "extra_probe_peers": [], 2024-05-08T23:40:58.243 INFO:teuthology.orchestra.run.smithi033.stdout: "sync_provider": [], 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "monmap": { 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 1, 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "fsid": "7b439b97-64c0-4aab-ac3b-1b32ed376bfb", 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "modified": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "created": "2024-05-08T23:31:19.072891Z", 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release": 18, 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min_mon_release_name": "reef", 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "election_strategy": 1, 2024-05-08T23:40:58.244 INFO:teuthology.orchestra.run.smithi033.stdout: "disallowed_leaders: ": "", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false, 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tiebreaker_mon": "", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "removed_ranks: ": "", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "features": { 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "persistent": [ 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "kraken", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "luminous", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "mimic", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap-prune", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "nautilus", 2024-05-08T23:40:58.245 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus", 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific", 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "elector-pinging", 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy", 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "reef" 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "optional": [] 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "mons": [ 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 0, 2024-05-08T23:40:58.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "a", 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:3300", 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789", 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.249 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6789/0", 2024-05-08T23:40:58.249 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.33:6789/0", 2024-05-08T23:40:58.249 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 1, 2024-05-08T23:40:58.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "b", 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3300", 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789", 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6789/0", 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6789/0", 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: "rank": 2, 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "c", 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addrs": { 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T23:40:58.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:3301", 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T23:40:58.255 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790", 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 0 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.42:6790/0", 2024-05-08T23:40:58.256 INFO:teuthology.orchestra.run.smithi033.stdout: "public_addr": "172.21.15.42:6790/0", 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: "priority": 0, 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: "weight": 0, 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_location": "{}" 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: "feature_map": { 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": [ 2024-05-08T23:40:58.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 1 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": [ 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:58.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": [ 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 6 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client": [ 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.259 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 3 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": [ 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: "features": "0x3f01cfbffffdffff", 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: "release": "luminous", 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 2 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T23:40:58.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T23:40:58.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T23:40:58.261 INFO:teuthology.orchestra.run.smithi033.stdout: "stretch_mode": false 2024-05-08T23:40:58.261 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:40:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f589406acc0 msgr2=0x7f589406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f589406acc0 0x7f589406d180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58b414b320 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 msgr2=0x7f58b41efe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b37fe700 1 -- 172.21.15.33:0/1104831435 reap_dead start 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f58b4073780 tx=0x7f58ac004ad0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 shutdown_connections 2024-05-08T23:40:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 --2- 172.21.15.33:0/1104831435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b4154770 0x7f58b41efe80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 >> 172.21.15.33:0/1104831435 conn(0x7f58b4066ae0 msgr2=0x7f58b41534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 shutdown_connections 2024-05-08T23:40:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.242+0000 7f58b94e9700 1 -- 172.21.15.33:0/1104831435 wait complete. 2024-05-08T23:40:58.265 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-08T23:40:58.266 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T23:40:58.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T23:40:58.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.407+0000 7f9e05890700 1 Processor -- start 2024-05-08T23:40:58.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.407+0000 7f9e05890700 1 -- start start 2024-05-08T23:40:58.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9e05890700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9e00165c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9e05890700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0006e7c0 con 0x7f9e0006ec00 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9e05890700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0006e200 con 0x7f9e00154280 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9e05890700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0006f0f0 con 0x7f9e000b3000 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9dfe7fc700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e0006ec00 0x7f9e000ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:56700/0 (socket says 172.21.15.33:56700) 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.408+0000 7f9dfe7fc700 1 -- 172.21.15.33:0/1710921607 learned_addr learned my addr 172.21.15.33:0/1710921607 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dff7fe700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9e00165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746565649 0 0) 0x7f9e0006e7c0 con 0x7f9e0006ec00 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec0035b0 con 0x7f9e0006ec00 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686803818 0 0) 0x7f9dec0035b0 con 0x7f9e0006ec00 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 msgr2=0x7f9e00165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9e00165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 >> v1:172.21.15.42:6789/0 conn(0x7f9e00154280 legacy=0x7f9e0015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0006e4e0 con 0x7f9e0006ec00 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f9de8001670 con 0x7f9e0006ec00 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 >> v1:172.21.15.33:6789/0 conn(0x7f9e0006ec00 legacy=0x7f9e000ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9dec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9dec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9dec004d90 0x7f9dec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0006f0f0 con 0x7f9dec003a30 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec00bac0 con 0x7f9dec004d90 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec00bc80 con 0x7f9e000b3000 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9dec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dfe7fc700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9dec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.409+0000 7f9dff7fe700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9dec004d90 0x7f9dec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 msgr2=0x7f9dec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e000b3000 0x7f9dec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9dec004d90 msgr2=0x7f9dec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9dec004d90 0x7f9dec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1710921607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0035b0 con 0x7f9dec003a30 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9dec003e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9df00018d0 tx=0x7f9dec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1bd1a93a54b90f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000c690 con 0x7f9dec003a30 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000c810 con 0x7f9dec003a30 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.410+0000 7f9dfdffb700 1 -- 172.21.15.33:0/1710921607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000cbc0 con 0x7f9dec003a30 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 -- 172.21.15.33:0/1710921607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 msgr2=0x7f9dec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 --2- 172.21.15.33:0/1710921607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9dec003e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9df00018d0 tx=0x7f9dec005880 comp rx=0 tx=0).stop 2024-05-08T23:40:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9dff7fe700 1 -- 172.21.15.33:0/1710921607 reap_dead start 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 -- 172.21.15.33:0/1710921607 shutdown_connections 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 -- 172.21.15.33:0/1710921607 >> 172.21.15.33:0/1710921607 conn(0x7f9e00066ae0 msgr2=0x7f9e00066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 -- 172.21.15.33:0/1710921607 shutdown_connections 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.411+0000 7f9e05890700 1 -- 172.21.15.33:0/1710921607 wait complete. 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.412+0000 7f9e05890700 1 Processor -- start 2024-05-08T23:40:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.412+0000 7f9e05890700 1 -- start start 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e001503a0 0x7f9e0014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 0x7f9e00147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0015a7c0 con 0x7f9dec003a30 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001f02c0 con 0x7f9e0014c210 2024-05-08T23:40:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9e05890700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0014f260 con 0x7f9e001503a0 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40120/0 (socket says 172.21.15.33:40120) 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1136458036 learned_addr learned my addr 172.21.15.33:0/1136458036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dff7fe700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 0x7f9e00147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfe7fc700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e001503a0 0x7f9e0014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e001503a0 msgr2=0x7f9e0014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e001503a0 0x7f9e0014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 msgr2=0x7f9e00147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 0x7f9e00147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.413+0000 7f9dfeffd700 1 -- 172.21.15.33:0/1136458036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0014c690 con 0x7f9dec003a30 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9dff7fe700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 0x7f9e00147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9dfeffd700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9df000c0e0 tx=0x7f9df0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000cb80 con 0x7f9dec003a30 2024-05-08T23:40:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0021070 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e001e3ee0 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000cee0 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.414+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0014e650 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.415+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e00075460 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.415+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9df0023030 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.416+0000 7f9df7fff700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.416+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9df008a6d0 con 0x7f9dec003a30 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.416+0000 7f9dfe7fc700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:40:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.416+0000 7f9dfe7fc700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 0x7f9de006d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e0015e490 tx=0x7f9de8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:40:58.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.421+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f9df00557d0 con 0x7f9dec003a30 2024-05-08T23:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.742+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f9e0006bec0 con 0x7f9dec003a30 2024-05-08T23:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.743+0000 7f9df7fff700 1 -- 172.21.15.33:0/1136458036 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f9df0058df0 con 0x7f9dec003a30 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 msgr2=0x7f9de006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 0x7f9de006d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e0015e490 tx=0x7f9de8009040 comp rx=0 tx=0).stop 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 msgr2=0x7f9e00150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9df000c0e0 tx=0x7f9df0001230 comp rx=0 tx=0).stop 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 shutdown_connections 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9de006ac90 0x7f9de006d150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e0014c210 0x7f9e00147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec003a30 0x7f9e00150d80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 --2- 172.21.15.33:0/1136458036 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9e001503a0 0x7f9e0014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.747+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 >> 172.21.15.33:0/1136458036 conn(0x7f9e00066ae0 msgr2=0x7f9e001587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.748+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 shutdown_connections 2024-05-08T23:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:40:58.748+0000 7f9e05890700 1 -- 172.21.15.33:0/1136458036 wait complete. 2024-05-08T23:40:58.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-08T23:41:08.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:41:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.909+0000 7f3330e93700 1 Processor -- start 2024-05-08T23:41:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.909+0000 7f3330e93700 1 -- start start 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c1511c0 0x7f332c072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f332c0b2b60 0x7f332c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c06ce60 con 0x7f332c1511c0 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c15b470 con 0x7f332c0b2b60 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3330e93700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0c90a0 con 0x7f332c15af30 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c1511c0 0x7f332c072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f332c1511c0 0x7f332c072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44172/0 (socket says 172.21.15.33:44172) 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 -- 172.21.15.33:0/3171736356 learned_addr learned my addr 172.21.15.33:0/3171736356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:08.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3323fff700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332ad9d700 1 --2- 172.21.15.33:0/3171736356 >> v2:172.21.15.42:3300/0 conn(0x7f332c0b2b60 0x7f332c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 -- 172.21.15.33:0/3171736356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 msgr2=0x7f332c072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 -- 172.21.15.33:0/3171736356 >> v2:172.21.15.42:3300/0 conn(0x7f332c0b2b60 msgr2=0x7f332c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 --2- 172.21.15.33:0/3171736356 >> v2:172.21.15.42:3300/0 conn(0x7f332c0b2b60 0x7f332c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 -- 172.21.15.33:0/3171736356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0c9380 con 0x7f332c1511c0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f332a59c700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c072400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f332c0c2b20 tx=0x7f331c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abd49279726faa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.911+0000 7f3329d9b700 1 -- 172.21.15.33:0/3171736356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c00b870 con 0x7f332c1511c0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3329d9b700 1 -- 172.21.15.33:0/3171736356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c00b9d0 con 0x7f332c1511c0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3329d9b700 1 -- 172.21.15.33:0/3171736356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c00bd60 con 0x7f332c1511c0 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 -- 172.21.15.33:0/3171736356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 msgr2=0x7f332c072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c072400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f332c0c2b20 tx=0x7f331c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 -- 172.21.15.33:0/3171736356 shutdown_connections 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c072400 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 --2- 172.21.15.33:0/3171736356 >> v2:172.21.15.42:3300/0 conn(0x7f332c0b2b60 0x7f332c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 --2- 172.21.15.33:0/3171736356 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 -- 172.21.15.33:0/3171736356 >> 172.21.15.33:0/3171736356 conn(0x7f332c06e990 msgr2=0x7f332c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 -- 172.21.15.33:0/3171736356 shutdown_connections 2024-05-08T23:41:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.912+0000 7f3330e93700 1 -- 172.21.15.33:0/3171736356 wait complete. 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 Processor -- start 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 -- start start 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c0b2b60 0x7f332c155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c1eed10 con 0x7f332c1511c0 2024-05-08T23:41:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c1551c0 con 0x7f332c0b2b60 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.914+0000 7f3330e93700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c1554a0 con 0x7f332c15af30 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44182/0 (socket says 172.21.15.33:44182) 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 -- 172.21.15.33:0/4040323122 learned_addr learned my addr 172.21.15.33:0/4040323122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f332a59c700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c0b2b60 0x7f332c155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f332ad9d700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 -- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 msgr2=0x7f332c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 -- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c0b2b60 msgr2=0x7f332c155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c0b2b60 0x7f332c155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 -- 172.21.15.33:0/4040323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c009550 con 0x7f332c1511c0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3323fff700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f331400e810 tx=0x7f331400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314013610 con 0x7f332c1511c0 2024-05-08T23:41:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3314013770 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c14b060 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.915+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314013b00 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.916+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c14b620 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.917+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c075460 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.918+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3314013c60 con 0x7f332c1511c0 2024-05-08T23:41:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.918+0000 7f3322ffd700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 0x7f330c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:08.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.918+0000 7f332a59c700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 0x7f330c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:08.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.918+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3314089500 con 0x7f332c1511c0 2024-05-08T23:41:08.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.919+0000 7f332a59c700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 0x7f330c06d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f331c000c00 tx=0x7f331c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:08.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:08.923+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3314054660 con 0x7f332c1511c0 2024-05-08T23:41:09.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.190+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332c14a6e0 con 0x7f332c1511c0 2024-05-08T23:41:09.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.191+0000 7f3322ffd700 1 -- 172.21.15.33:0/4040323122 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7f3314017090 con 0x7f332c1511c0 2024-05-08T23:41:09.189 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:09.189 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 msgr2=0x7f330c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 0x7f330c06d180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f331c000c00 tx=0x7f331c008030 comp rx=0 tx=0).stop 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 msgr2=0x7f332c1ee7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f331400e810 tx=0x7f331400ebd0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 shutdown_connections 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f330c06acc0 0x7f330c06d180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f332c15af30 0x7f332c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f332c0b2b60 0x7f332c155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 --2- 172.21.15.33:0/4040323122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f332c1511c0 0x7f332c1ee7d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 >> 172.21.15.33:0/4040323122 conn(0x7f332c06e990 msgr2=0x7f332c06cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 shutdown_connections 2024-05-08T23:41:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.193+0000 7f3330e93700 1 -- 172.21.15.33:0/4040323122 wait complete. 2024-05-08T23:41:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:41:09.205 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-08T23:41:09.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T23:41:09.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.344+0000 7f1076178700 1 Processor -- start 2024-05-08T23:41:09.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.344+0000 7f1076178700 1 -- start start 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f1076178700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10701572c0 0x7f10700cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f1076178700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700c3580 con 0x7f10700cc5d0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f1076178700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700c2fa0 con 0x7f10701576a0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f1076178700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700cc430 con 0x7f10701572c0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106f7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10701572c0 0x7f10700cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106effd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f10700cc5d0 0x7f10700c3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40192/0 (socket says 172.21.15.33:40192) 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106effd700 1 -- 172.21.15.33:0/3487028728 learned_addr learned my addr 172.21.15.33:0/3487028728 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106f7fe700 1 -- 172.21.15.33:0/3487028728 >> v1:172.21.15.42:6789/0 conn(0x7f10701576a0 legacy=0x7f1070150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106f7fe700 1 -- 172.21.15.33:0/3487028728 >> v1:172.21.15.33:6789/0 conn(0x7f10700cc5d0 legacy=0x7f10700c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106f7fe700 1 -- 172.21.15.33:0/3487028728 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700ccaa0 con 0x7f10701572c0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.345+0000 7f106e7fc700 1 -- 172.21.15.33:0/3487028728 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465437550 0 0) 0x7f10700c3580 con 0x7f10700cc5d0 2024-05-08T23:41:09.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f106f7fe700 1 --2- 172.21.15.33:0/3487028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701572c0 0x7f10700cbef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279d15d85596845e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f106e7fc700 1 -- 172.21.15.33:0/3487028728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011830 con 0x7f10701572c0 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f106e7fc700 1 -- 172.21.15.33:0/3487028728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060011990 con 0x7f10701572c0 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f106e7fc700 1 -- 172.21.15.33:0/3487028728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011d20 con 0x7f10701572c0 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 -- 172.21.15.33:0/3487028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701572c0 msgr2=0x7f10700cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 --2- 172.21.15.33:0/3487028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701572c0 0x7f10700cbef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 -- 172.21.15.33:0/3487028728 shutdown_connections 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 --2- 172.21.15.33:0/3487028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701572c0 0x7f10700cbef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 -- 172.21.15.33:0/3487028728 >> 172.21.15.33:0/3487028728 conn(0x7f107006eb10 msgr2=0x7f107006ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 -- 172.21.15.33:0/3487028728 shutdown_connections 2024-05-08T23:41:09.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.346+0000 7f1076178700 1 -- 172.21.15.33:0/3487028728 wait complete. 2024-05-08T23:41:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.348+0000 7f1076178700 1 Processor -- start 2024-05-08T23:41:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.348+0000 7f1076178700 1 -- start start 2024-05-08T23:41:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10700cc5d0 0x7f1070071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10701572c0 0x7f1070154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107014e410 con 0x7f10701576a0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070072710 con 0x7f10700cc5d0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f1076178700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10701500e0 con 0x7f10701572c0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44204/0 (socket says 172.21.15.33:44204) 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 -- 172.21.15.33:0/503670019 learned_addr learned my addr 172.21.15.33:0/503670019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106f7fe700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10700cc5d0 0x7f1070071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106effd700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10701572c0 0x7f1070154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 -- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10701572c0 msgr2=0x7f1070154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10701572c0 0x7f1070154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 -- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10700cc5d0 msgr2=0x7f1070071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10700cc5d0 0x7f1070071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 -- 172.21.15.33:0/503670019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600094a0 con 0x7f10701576a0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.349+0000 7f106ffff700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106800cb80 tx=0x7f106800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.350+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068003c00 con 0x7f10701576a0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.350+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068003d60 con 0x7f10701576a0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.350+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1070072480 con 0x7f10701576a0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.350+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106800aaa0 con 0x7f10701576a0 2024-05-08T23:41:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.350+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107014f7b0 con 0x7f10701576a0 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.351+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070150290 con 0x7f10701576a0 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.351+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f106800ac00 con 0x7f10701576a0 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.351+0000 7f1057fff700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 0x7f105c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.352+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f10680893f0 con 0x7f10701576a0 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.352+0000 7f106f7fe700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 0x7f105c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.352+0000 7f106f7fe700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 0x7f105c06d150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10600090b0 tx=0x7f106001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.356+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1070150290 con 0x7f10701576a0 2024-05-08T23:41:09.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.688+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1070150250 con 0x7f10701576a0 2024-05-08T23:41:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.688+0000 7f1057fff700 1 -- 172.21.15.33:0/503670019 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f1068054470 con 0x7f10701576a0 2024-05-08T23:41:09.686 INFO:teuthology.orchestra.run.smithi033.stdout:{"election_epoch":74,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","modified":"2024-05-08T23:31:19.072891Z","created":"2024-05-08T23:31:19.072891Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3301","nonce":0},{"type":"v1","addr":"172.21.15.42:6790","nonce":0}]},"addr":"172.21.15.42:6790/0","public_addr":"172.21.15.42:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T23:41:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.690+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 msgr2=0x7f105c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 0x7f105c06d150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10600090b0 tx=0x7f106001c040 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 msgr2=0x7f107013d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f106800cb80 tx=0x7f106800ce90 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 shutdown_connections 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f105c06ac90 0x7f105c06d150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10701576a0 0x7f107013d6d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10700cc5d0 0x7f1070071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 --2- 172.21.15.33:0/503670019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10701572c0 0x7f1070154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 >> 172.21.15.33:0/503670019 conn(0x7f107006eb10 msgr2=0x7f10700c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 shutdown_connections 2024-05-08T23:41:09.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.691+0000 7f1076178700 1 -- 172.21.15.33:0/503670019 wait complete. 2024-05-08T23:41:09.702 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-08T23:41:09.702 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T23:41:09.712 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T23:41:09.712 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:41:09.712 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:41:09.731 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T23:41:09.731 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:41:09.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:41:09.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.981+0000 7f9165796700 1 Processor -- start 2024-05-08T23:41:09.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.981+0000 7f9165796700 1 -- start start 2024-05-08T23:41:09.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.982+0000 7f9165796700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.982+0000 7f9165796700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 0x7f91600c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.982+0000 7f9165796700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9160070450 con 0x7f91600c38a0 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.982+0000 7f9165796700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600c2e90 con 0x7f91600c85a0 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.982+0000 7f9165796700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9160070730 con 0x7f9160152350 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915f7fe700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 0x7f91600c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915e7fc700 1 --1- >> v1:172.21.15.42:6789/0 conn(0x7f91600c38a0 0x7f91600c9520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.42:6789/0 says I am v1:172.21.15.33:51200/0 (socket says 172.21.15.33:51200) 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3301/0 says I am v2:172.21.15.33:37044/0 (socket says 172.21.15.33:37044) 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915e7fc700 1 -- 172.21.15.33:0/1680537364 learned_addr learned my addr 172.21.15.33:0/1680537364 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 -- 172.21.15.33:0/1680537364 >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 msgr2=0x7f91600c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 --2- 172.21.15.33:0/1680537364 >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 0x7f91600c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 -- 172.21.15.33:0/1680537364 >> v1:172.21.15.42:6789/0 conn(0x7f91600c38a0 legacy=0x7f91600c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 -- 172.21.15.33:0/1680537364 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160070a10 con 0x7f9160152350 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915f7fe700 1 --2- 172.21.15.33:0/1680537364 >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 0x7f91600c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:09.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915effd700 1 --2- 172.21.15.33:0/1680537364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9150009840 tx=0x7f9150009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fed7c778182fae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915dffb700 1 -- 172.21.15.33:0/1680537364 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150011830 con 0x7f9160152350 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915dffb700 1 -- 172.21.15.33:0/1680537364 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150011990 con 0x7f9160152350 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.983+0000 7f915dffb700 1 -- 172.21.15.33:0/1680537364 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150011d20 con 0x7f9160152350 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 -- 172.21.15.33:0/1680537364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 msgr2=0x7f9160158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 --2- 172.21.15.33:0/1680537364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9150009840 tx=0x7f9150009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 -- 172.21.15.33:0/1680537364 shutdown_connections 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 --2- 172.21.15.33:0/1680537364 >> v2:172.21.15.42:3300/0 conn(0x7f91600c85a0 0x7f91600c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 --2- 172.21.15.33:0/1680537364 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9160152350 0x7f9160158110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 -- 172.21.15.33:0/1680537364 >> 172.21.15.33:0/1680537364 conn(0x7f91600666c0 msgr2=0x7f9160066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 -- 172.21.15.33:0/1680537364 shutdown_connections 2024-05-08T23:41:09.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.984+0000 7f9165796700 1 -- 172.21.15.33:0/1680537364 wait complete. 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.985+0000 7f9165796700 1 Processor -- start 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.985+0000 7f9165796700 1 -- start start 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 0x7f91600c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 0x7f916014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600c66c0 con 0x7f9160152350 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91601eddd0 con 0x7f91600c85a0 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f9165796700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91601edb70 con 0x7f91600c38a0 2024-05-08T23:41:09.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f915f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f915f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44214/0 (socket says 172.21.15.33:44214) 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f915f7fe700 1 -- 172.21.15.33:0/1830471898 learned_addr learned my addr 172.21.15.33:0/1830471898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f915e7fc700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 0x7f916014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.986+0000 7f915effd700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 0x7f91600c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 -- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 msgr2=0x7f91600c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 0x7f91600c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 -- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 msgr2=0x7f916014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 0x7f916014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 -- 172.21.15.33:0/1830471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009a70 con 0x7f9160152350 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915e7fc700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 0x7f916014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915effd700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 0x7f91600c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f915f7fe700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f915800efc0 tx=0x7f915800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915800ca90 con 0x7f9160152350 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915800f040 con 0x7f9160152350 2024-05-08T23:41:09.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91500094a0 con 0x7f9160152350 2024-05-08T23:41:09.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915800ce10 con 0x7f9160152350 2024-05-08T23:41:09.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.987+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91600c5d90 con 0x7f9160152350 2024-05-08T23:41:09.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.988+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91600c6870 con 0x7f9160152350 2024-05-08T23:41:09.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.989+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9158014480 con 0x7f9160152350 2024-05-08T23:41:09.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.989+0000 7f91577fe700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 0x7f914006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:09.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.989+0000 7f915effd700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 0x7f914006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:09.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.989+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9158089a50 con 0x7f9160152350 2024-05-08T23:41:09.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.990+0000 7f915effd700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 0x7f914006d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f9150008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:09.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:09.994+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f91600c6870 con 0x7f9160152350 2024-05-08T23:41:10.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.214+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91600c6870 con 0x7f9160152350 2024-05-08T23:41:10.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.215+0000 7f91577fe700 1 -- 172.21.15.33:0/1830471898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16379 (secure 0 0 0) 0x7f91600c6870 con 0x7f9160152350 2024-05-08T23:41:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:10.213 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:59.845491+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:41:10.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 msgr2=0x7f914006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 0x7f914006d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9150000c00 tx=0x7f9150008030 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 msgr2=0x7f916014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f915800efc0 tx=0x7f915800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 shutdown_connections 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f914006ac90 0x7f914006d150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9160152350 0x7f916014a770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f91600c38a0 0x7f91600c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.217+0000 7f9165796700 1 --2- 172.21.15.33:0/1830471898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91600c85a0 0x7f916014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.218+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 >> 172.21.15.33:0/1830471898 conn(0x7f91600666c0 msgr2=0x7f91600b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.218+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 shutdown_connections 2024-05-08T23:41:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.218+0000 7f9165796700 1 -- 172.21.15.33:0/1830471898 wait complete. 2024-05-08T23:41:10.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:32:04.830340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:32:08.545052+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:32:09.301342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:41:10.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:41:10.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.373+0000 7fd71ac9c700 1 Processor -- start 2024-05-08T23:41:10.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.373+0000 7fd71ac9c700 1 -- start start 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd71ac9c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd7140c8e30 0x7fd714158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd71ac9c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7140cdd30 con 0x7fd7140c8a50 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd71ac9c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd714158610 con 0x7fd7140c38a0 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd71ac9c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd714070410 con 0x7fd7140c8e30 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd718a38700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7140c8a50 0x7fd714151e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40198/0 (socket says 172.21.15.33:40198) 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd718a38700 1 -- 172.21.15.33:0/2698764874 learned_addr learned my addr 172.21.15.33:0/2698764874 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:10.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.374+0000 7fd719239700 1 --2- 172.21.15.33:0/2698764874 >> v2:172.21.15.42:3300/0 conn(0x7fd7140c8e30 0x7fd714158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377505350 0 0) 0x7fd7140cdd30 con 0x7fd7140c8a50 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7000035b0 con 0x7fd7140c8a50 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd719239700 1 -- 172.21.15.33:0/2698764874 >> v1:172.21.15.42:6789/0 conn(0x7fd7140c38a0 legacy=0x7fd714160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd719239700 1 -- 172.21.15.33:0/2698764874 >> v1:172.21.15.33:6789/0 conn(0x7fd7140c8a50 legacy=0x7fd714151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271785640 0 0) 0x7fd7000035b0 con 0x7fd7140c8a50 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd719239700 1 -- 172.21.15.33:0/2698764874 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714158fe0 con 0x7fd7140c8e30 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd719239700 1 --2- 172.21.15.33:0/2698764874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8e30 0x7fd714158aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009d90 tx=0x7fd70c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a45a7b61cf5ef64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00e070 con 0x7fd7140c8e30 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c003a20 con 0x7fd7140c8e30 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.375+0000 7fd7137fe700 1 -- 172.21.15.33:0/2698764874 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704001670 con 0x7fd7140c8e30 2024-05-08T23:41:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 -- 172.21.15.33:0/2698764874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8e30 msgr2=0x7fd714158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/2698764874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8e30 0x7fd714158aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009d90 tx=0x7fd70c00b330 comp rx=0 tx=0).stop 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 -- 172.21.15.33:0/2698764874 shutdown_connections 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/2698764874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8e30 0x7fd714158aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 -- 172.21.15.33:0/2698764874 >> 172.21.15.33:0/2698764874 conn(0x7fd714072230 msgr2=0x7fd714072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 -- 172.21.15.33:0/2698764874 shutdown_connections 2024-05-08T23:41:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.376+0000 7fd71ac9c700 1 -- 172.21.15.33:0/2698764874 wait complete. 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.377+0000 7fd71ac9c700 1 Processor -- start 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.377+0000 7fd71ac9c700 1 -- start start 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 0x7fd7140c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7140c8e30 0x7fd7140c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140c6640 con 0x7fd7140c38a0 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7140c56e0 con 0x7fd7140c8a50 2024-05-08T23:41:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd71ac9c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7141ee630 con 0x7fd7140c8e30 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd718a38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd718a38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44224/0 (socket says 172.21.15.33:44224) 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.378+0000 7fd718a38700 1 -- 172.21.15.33:0/3990391841 learned_addr learned my addr 172.21.15.33:0/3990391841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd713fff700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 0x7fd7140c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd719239700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7140c8e30 0x7fd7140c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 -- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7140c8e30 msgr2=0x7fd7140c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7140c8e30 0x7fd7140c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 -- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 msgr2=0x7fd7140c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 0x7fd7140c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 -- 172.21.15.33:0/3990391841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c009a70 con 0x7fd7140c38a0 2024-05-08T23:41:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd713fff700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 0x7fd7140c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd718a38700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7040011e0 tx=0x7fd70400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704007560 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7040076c0 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704004a90 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7140c5980 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.379+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7140c5c60 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.380+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd714075460 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.381+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd704004bf0 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.381+0000 7fd7117fa700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 0x7fd6f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.381+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd704088550 con 0x7fd7140c38a0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.381+0000 7fd713fff700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 0x7fd6f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.382+0000 7fd713fff700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 0x7fd6f406d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd71414db40 tx=0x7fd6fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.386+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd704053660 con 0x7fd7140c38a0 2024-05-08T23:41:10.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.605+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd71414c000 con 0x7fd7140c38a0 2024-05-08T23:41:10.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.605+0000 7fd7117fa700 1 -- 172.21.15.33:0/3990391841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd704056c80 con 0x7fd7140c38a0 2024-05-08T23:41:10.603 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T23:41:10.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 msgr2=0x7fd6f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 0x7fd6f406d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd71414db40 tx=0x7fd6fc009040 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 msgr2=0x7fd71414d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7040011e0 tx=0x7fd70400cb00 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 shutdown_connections 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd6f406ac90 0x7fd6f406d150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7140c8e30 0x7fd7140c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7140c38a0 0x7fd71414d160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 --2- 172.21.15.33:0/3990391841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7140c8a50 0x7fd7140c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 >> 172.21.15.33:0/3990391841 conn(0x7fd714072230 msgr2=0x7fd71414c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 shutdown_connections 2024-05-08T23:41:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.608+0000 7fd71ac9c700 1 -- 172.21.15.33:0/3990391841 wait complete. 2024-05-08T23:41:10.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:41:10.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.765+0000 7fa3f6d70700 1 Processor -- start 2024-05-08T23:41:10.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.765+0000 7fa3f6d70700 1 -- start start 2024-05-08T23:41:10.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f6d70700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fa3f015e3d0 0x7fa3f00ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f6d70700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f006e1e0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f6d70700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f006e720 con 0x7fa3f0154480 2024-05-08T23:41:10.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f6d70700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f006e440 con 0x7fa3f015e3d0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3effff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa3f00b30b0 0x7fa3f006d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40214/0 (socket says 172.21.15.33:40214) 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3effff700 1 -- 172.21.15.33:0/2359842775 learned_addr learned my addr 172.21.15.33:0/2359842775 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f530d700 1 --2- 172.21.15.33:0/2359842775 >> v2:172.21.15.42:3300/0 conn(0x7fa3f015e3d0 0x7fa3f00ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851006016 0 0) 0x7fa3f006e1e0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e00035b0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f530d700 1 -- 172.21.15.33:0/2359842775 >> v1:172.21.15.42:6789/0 conn(0x7fa3f0154480 legacy=0x7fa3f006eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939856921 0 0) 0x7fa3e00035b0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f530d700 1 -- 172.21.15.33:0/2359842775 >> v1:172.21.15.33:6789/0 conn(0x7fa3f00b30b0 legacy=0x7fa3f006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.766+0000 7fa3f530d700 1 -- 172.21.15.33:0/2359842775 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f015d8f0 con 0x7fa3f015e3d0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3f530d700 1 --2- 172.21.15.33:0/2359842775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f00ca8c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009d90 tx=0x7fa3e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b16cfcf40f00064 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800e070 con 0x7fa3f015e3d0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8003a20 con 0x7fa3f015e3d0 2024-05-08T23:41:10.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3ef7fe700 1 -- 172.21.15.33:0/2359842775 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc001670 con 0x7fa3f015e3d0 2024-05-08T23:41:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3f6d70700 1 -- 172.21.15.33:0/2359842775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 msgr2=0x7fa3f00ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.767+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/2359842775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f00ca8c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009d90 tx=0x7fa3e800b330 comp rx=0 tx=0).stop 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.768+0000 7fa3f6d70700 1 -- 172.21.15.33:0/2359842775 shutdown_connections 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.768+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/2359842775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f00ca8c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.768+0000 7fa3f6d70700 1 -- 172.21.15.33:0/2359842775 >> 172.21.15.33:0/2359842775 conn(0x7fa3f0072670 msgr2=0x7fa3f0072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.768+0000 7fa3f6d70700 1 -- 172.21.15.33:0/2359842775 shutdown_connections 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.768+0000 7fa3f6d70700 1 -- 172.21.15.33:0/2359842775 wait complete. 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.769+0000 7fa3f6d70700 1 Processor -- start 2024-05-08T23:41:10.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.769+0000 7fa3f6d70700 1 -- start start 2024-05-08T23:41:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa3f0154480 0x7fa3f01efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f01f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f3250 con 0x7fa3f00b30b0 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0148f00 con 0x7fa3f015e3d0 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f6d70700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c82a0 con 0x7fa3f0154480 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f4b0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f4b0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44240/0 (socket says 172.21.15.33:44240) 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f4b0c700 1 -- 172.21.15.33:0/3817029955 learned_addr learned my addr 172.21.15.33:0/3817029955 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3f530d700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f01f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.770+0000 7fa3effff700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa3f0154480 0x7fa3f01efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 -- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa3f0154480 msgr2=0x7fa3f01efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa3f0154480 0x7fa3f01efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 -- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 msgr2=0x7fa3f01f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:10.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f01f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 -- 172.21.15.33:0/3817029955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009a70 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f4b0c700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c550 tx=0x7fa3e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40054e0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e4003680 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f01f34a0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003a30 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.771+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f01f3a60 con 0x7fa3f00b30b0 2024-05-08T23:41:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.772+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f0075460 con 0x7fa3f00b30b0 2024-05-08T23:41:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.773+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fa3e4003b90 con 0x7fa3f00b30b0 2024-05-08T23:41:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.773+0000 7fa3ed7fa700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 0x7fa3d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.773+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fa3e4088730 con 0x7fa3f00b30b0 2024-05-08T23:41:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.773+0000 7fa3effff700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 0x7fa3d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:10.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.774+0000 7fa3effff700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 0x7fa3d406d150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0144850 tx=0x7fa3dc001da0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:10.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.778+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fa3e40538d0 con 0x7fa3f00b30b0 2024-05-08T23:41:10.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.999+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3f006e970 con 0x7fa3f00b30b0 2024-05-08T23:41:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:10.999+0000 7fa3ed7fa700 1 -- 172.21.15.33:0/3817029955 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3e4017020 con 0x7fa3f00b30b0 2024-05-08T23:41:10.997 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T23:41:10.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 msgr2=0x7fa3d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 0x7fa3d406d150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0144850 tx=0x7fa3dc001da0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 msgr2=0x7fa3f015c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c550 tx=0x7fa3e400c860 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 shutdown_connections 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fa3d406ac90 0x7fa3d406d150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3f015e3d0 0x7fa3f01f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f00b30b0 0x7fa3f015c8f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 --2- 172.21.15.33:0/3817029955 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fa3f0154480 0x7fa3f01efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 >> 172.21.15.33:0/3817029955 conn(0x7fa3f0072670 msgr2=0x7fa3f00c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 shutdown_connections 2024-05-08T23:41:11.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.002+0000 7fa3f6d70700 1 -- 172.21.15.33:0/3817029955 wait complete. 2024-05-08T23:41:11.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:41:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.156+0000 7f3c43d4d700 1 Processor -- start 2024-05-08T23:41:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 -- start start 2024-05-08T23:41:11.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c3c1546c0 0x7f3c3c072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3c3c0c7420 0x7f3c3c06b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c06bb70 con 0x7f3c3c15d9d0 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c0719b0 con 0x7f3c3c1546c0 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.157+0000 7f3c43d4d700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c0c7860 con 0x7f3c3c0c7420 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c3c1546c0 0x7f3c3c072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3c3c1546c0 0x7f3c3c072ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44254/0 (socket says 172.21.15.33:44254) 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3971738300 learned_addr learned my addr 172.21.15.33:0/3971738300 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c412e8700 1 --2- 172.21.15.33:0/3971738300 >> v2:172.21.15.42:3300/0 conn(0x7f3c3c0c7420 0x7f3c3c06b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3971738300 >> v2:172.21.15.42:3300/0 conn(0x7f3c3c0c7420 msgr2=0x7f3c3c06b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 --2- 172.21.15.33:0/3971738300 >> v2:172.21.15.42:3300/0 conn(0x7f3c3c0c7420 0x7f3c3c06b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3971738300 >> v1:172.21.15.42:6789/0 conn(0x7f3c3c15d9d0 legacy=0x7f3c3c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3971738300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c06b8b0 con 0x7f3c3c1546c0 2024-05-08T23:41:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c41ae9700 1 --2- 172.21.15.33:0/3971738300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1546c0 0x7f3c3c072ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cdb70 tx=0x7f3c38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eac52e5ac68fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c40ae7700 1 -- 172.21.15.33:0/3971738300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800b870 con 0x7f3c3c1546c0 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c40ae7700 1 -- 172.21.15.33:0/3971738300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800b9d0 con 0x7f3c3c1546c0 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.158+0000 7f3c40ae7700 1 -- 172.21.15.33:0/3971738300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800bd60 con 0x7f3c3c1546c0 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3971738300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1546c0 msgr2=0x7f3c3c072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3971738300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1546c0 0x7f3c3c072ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cdb70 tx=0x7f3c38009bd0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3971738300 shutdown_connections 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3971738300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c1546c0 0x7f3c3c072ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3971738300 >> v2:172.21.15.42:3300/0 conn(0x7f3c3c0c7420 0x7f3c3c06b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3971738300 >> 172.21.15.33:0/3971738300 conn(0x7f3c3c0c9620 msgr2=0x7f3c3c0c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3971738300 shutdown_connections 2024-05-08T23:41:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.159+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3971738300 wait complete. 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.161+0000 7f3c43d4d700 1 Processor -- start 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.161+0000 7f3c43d4d700 1 -- start start 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c3c1546c0 0x7f3c3c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c3c15d9d0 0x7f3c3c14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c1f1320 con 0x7f3c3c0c7420 2024-05-08T23:41:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c14e760 con 0x7f3c3c15d9d0 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c43d4d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0c5cf0 con 0x7f3c3c1546c0 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44270/0 (socket says 172.21.15.33:44270) 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3390957369 learned_addr learned my addr 172.21.15.33:0/3390957369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c412e8700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c3c1546c0 0x7f3c3c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c422ea700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c3c15d9d0 0x7f3c3c14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c3c1546c0 msgr2=0x7f3c3c14dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c3c1546c0 0x7f3c3c14dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c3c15d9d0 msgr2=0x7f3c3c14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c3c15d9d0 0x7f3c3c14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.162+0000 7f3c41ae9700 1 -- 172.21.15.33:0/3390957369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009550 con 0x7f3c3c0c7420 2024-05-08T23:41:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c41ae9700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38011830 con 0x7f3c3c0c7420 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3801e070 con 0x7f3c3c0c7420 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c0c5f20 con 0x7f3c3c0c7420 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38011bd0 con 0x7f3c3c0c7420 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.163+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c1f1040 con 0x7f3c3c0c7420 2024-05-08T23:41:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.164+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c075460 con 0x7f3c3c0c7420 2024-05-08T23:41:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.164+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3c38011d30 con 0x7f3c3c0c7420 2024-05-08T23:41:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.165+0000 7f3c327fc700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 0x7f3c2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.165+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3c38020070 con 0x7f3c3c0c7420 2024-05-08T23:41:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.165+0000 7f3c412e8700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 0x7f3c2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.165+0000 7f3c412e8700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 0x7f3c2006d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0c4b10 tx=0x7f3c2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.169+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3c38055630 con 0x7f3c3c0c7420 2024-05-08T23:41:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.387+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c3c06bd00 con 0x7f3c3c0c7420 2024-05-08T23:41:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.387+0000 7f3c327fc700 1 -- 172.21.15.33:0/3390957369 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c38058c50 con 0x7f3c3c0c7420 2024-05-08T23:41:11.385 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T23:41:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 msgr2=0x7f3c2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 0x7f3c2006d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0c4b10 tx=0x7f3c2c008040 comp rx=0 tx=0).stop 2024-05-08T23:41:11.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 msgr2=0x7f3c3c0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c38004750 comp rx=0 tx=0).stop 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 shutdown_connections 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3c2006ac90 0x7f3c2006d150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c3c15d9d0 0x7f3c3c14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c0c7420 0x7f3c3c0c5370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 --2- 172.21.15.33:0/3390957369 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3c3c1546c0 0x7f3c3c14dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 >> 172.21.15.33:0/3390957369 conn(0x7f3c3c0c9620 msgr2=0x7f3c3c070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 shutdown_connections 2024-05-08T23:41:11.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.390+0000 7f3c43d4d700 1 -- 172.21.15.33:0/3390957369 wait complete. 2024-05-08T23:41:11.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:41:11.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.543+0000 7fbd51bb6700 1 Processor -- start 2024-05-08T23:41:11.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.543+0000 7fbd51bb6700 1 -- start start 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd51bb6700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbd4c154440 0x7fbd4c070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd51bb6700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c15e750 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd51bb6700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c0709a0 con 0x7fbd4c0c7210 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd51bb6700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4c15dbd0 con 0x7fbd4c154440 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4bfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbd4c154440 0x7fbd4c070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4b7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd4c0c6e30 0x7fbd4c15ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40230/0 (socket says 172.21.15.33:40230) 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4b7fe700 1 -- 172.21.15.33:0/4188229854 learned_addr learned my addr 172.21.15.33:0/4188229854 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4bfff700 1 -- 172.21.15.33:0/4188229854 >> v1:172.21.15.42:6789/0 conn(0x7fbd4c0c7210 legacy=0x7fbd4c153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4bfff700 1 -- 172.21.15.33:0/4188229854 >> v1:172.21.15.33:6789/0 conn(0x7fbd4c0c6e30 legacy=0x7fbd4c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4bfff700 1 -- 172.21.15.33:0/4188229854 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c15deb0 con 0x7fbd4c154440 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.544+0000 7fbd4affd700 1 -- 172.21.15.33:0/4188229854 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864559550 0 0) 0x7fbd4c15e750 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd4bfff700 1 --2- 172.21.15.33:0/4188229854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c154440 0x7fbd4c070e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa3cc8485024f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd4affd700 1 -- 172.21.15.33:0/4188229854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400e070 con 0x7fbd4c154440 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd4affd700 1 -- 172.21.15.33:0/4188229854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4400bad0 con 0x7fbd4c154440 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd4affd700 1 -- 172.21.15.33:0/4188229854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4400be80 con 0x7fbd4c154440 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 -- 172.21.15.33:0/4188229854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c154440 msgr2=0x7fbd4c070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/4188229854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c154440 0x7fbd4c070e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4400b330 comp rx=0 tx=0).stop 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 -- 172.21.15.33:0/4188229854 shutdown_connections 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/4188229854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c154440 0x7fbd4c070e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 -- 172.21.15.33:0/4188229854 >> 172.21.15.33:0/4188229854 conn(0x7fbd4c06a990 msgr2=0x7fbd4c06ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:11.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 -- 172.21.15.33:0/4188229854 shutdown_connections 2024-05-08T23:41:11.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.545+0000 7fbd51bb6700 1 -- 172.21.15.33:0/4188229854 wait complete. 2024-05-08T23:41:11.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.547+0000 7fbd51bb6700 1 Processor -- start 2024-05-08T23:41:11.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.547+0000 7fbd51bb6700 1 -- start start 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 0x7fbd4c0c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c154440 0x7fbd4c14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c14ba00 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0c5140 con 0x7fbd4c154440 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd51bb6700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0c5400 con 0x7fbd4c0c7210 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44296/0 (socket says 172.21.15.33:44296) 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 -- 172.21.15.33:0/2129683170 learned_addr learned my addr 172.21.15.33:0/2129683170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4bfff700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c154440 0x7fbd4c14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd42ffd700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 0x7fbd4c0c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 -- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 msgr2=0x7fbd4c0c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 0x7fbd4c0c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 -- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c154440 msgr2=0x7fbd4c14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c154440 0x7fbd4c14be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 -- 172.21.15.33:0/2129683170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009a70 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd42ffd700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 0x7fbd4c0c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.548+0000 7fbd4b7fe700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0011e0 tx=0x7fbd3c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.549+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004c80 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.549+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c004de0 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.549+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c0c56e0 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.549+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c007310 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.549+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c0c5ca0 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.550+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c153480 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.550+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fbd3c007470 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.551+0000 7fbd48ff9700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 0x7fbd2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.551+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fbd3c089100 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.551+0000 7fbd42ffd700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 0x7fbd2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:11.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.551+0000 7fbd42ffd700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 0x7fbd2c06d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c157bd0 tx=0x7fbd34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:11.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.555+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fbd3c054210 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.792+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd4c075460 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.792+0000 7fbd48ff9700 1 -- 172.21.15.33:0/2129683170 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd3c057830 con 0x7fbd4c0c6e30 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 msgr2=0x7fbd2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 0x7fbd2c06d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c157bd0 tx=0x7fbd34009040 comp rx=0 tx=0).stop 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 msgr2=0x7fbd4c1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0011e0 tx=0x7fbd3c00ca30 comp rx=0 tx=0).stop 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 shutdown_connections 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fbd2c06ac90 0x7fbd2c06d150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c154440 0x7fbd4c14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4c0c6e30 0x7fbd4c1583d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 --2- 172.21.15.33:0/2129683170 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fbd4c0c7210 0x7fbd4c0c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 >> 172.21.15.33:0/2129683170 conn(0x7fbd4c06a990 msgr2=0x7fbd4c152a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 shutdown_connections 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:11.795+0000 7fbd51bb6700 1 -- 172.21.15.33:0/2129683170 wait complete. 2024-05-08T23:41:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-05-08T23:41:11.807 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T23:41:11.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:41:11.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T23:41:11.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:41:11.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:41:11.997 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:41:12.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:41:12.058 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T23:41:12.084 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:41:12.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:41:12.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:41:12.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:41:12.166 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-08T23:41:12.270 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T23:41:12.283 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T23:41:12.284 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:41:12.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.429+0000 7f10cc6aa700 1 Processor -- start 2024-05-08T23:41:12.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.429+0000 7f10cc6aa700 1 -- start start 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10c40c2180 0x7f10c40c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c4071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c4070010 con 0x7f10c40c8e50 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c40c2aa0 con 0x7f10c40c2180 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4070870 con 0x7f10c406b660 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10c40c2180 0x7f10c40c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f10c40c2180 0x7f10c40c2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44308/0 (socket says 172.21.15.33:44308) 2024-05-08T23:41:12.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 -- 172.21.15.33:0/3511897487 learned_addr learned my addr 172.21.15.33:0/3511897487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cac47700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c4071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 -- 172.21.15.33:0/3511897487 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 msgr2=0x7f10c4071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c4071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 -- 172.21.15.33:0/3511897487 >> v1:172.21.15.42:6789/0 conn(0x7f10c40c8e50 legacy=0x7f10c4070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 -- 172.21.15.33:0/3511897487 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c415a820 con 0x7f10c40c2180 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10ca446700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c2560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098f0 tx=0x7f10c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9423c35b1f506ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10c9444700 1 -- 172.21.15.33:0/3511897487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000b8a0 con 0x7f10c40c2180 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10c9444700 1 -- 172.21.15.33:0/3511897487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000ba00 con 0x7f10c40c2180 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10c9444700 1 -- 172.21.15.33:0/3511897487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000bd90 con 0x7f10c40c2180 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3511897487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 msgr2=0x7f10c40c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c2560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098f0 tx=0x7f10c0009c00 comp rx=0 tx=0).stop 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3511897487 shutdown_connections 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c4071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3511897487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c2560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3511897487 >> 172.21.15.33:0/3511897487 conn(0x7f10c40c8360 msgr2=0x7f10c40c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3511897487 shutdown_connections 2024-05-08T23:41:12.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.430+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3511897487 wait complete. 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.432+0000 7f10cc6aa700 1 Processor -- start 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.432+0000 7f10cc6aa700 1 -- start start 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c40cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10c40c8e50 0x7f10c4145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41e30a0 con 0x7f10c40c2180 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40cb210 con 0x7f10c40c8e50 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cc6aa700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41573a0 con 0x7f10c406b660 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44312/0 (socket says 172.21.15.33:44312) 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 -- 172.21.15.33:0/3488607143 learned_addr learned my addr 172.21.15.33:0/3488607143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10ca446700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c40cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10cac47700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10c40c8e50 0x7f10c4145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 -- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 msgr2=0x7f10c40cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c40cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 -- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10c40c8e50 msgr2=0x7f10c4145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10c40c8e50 0x7f10c4145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.433+0000 7f10c9c45700 1 -- 172.21.15.33:0/3488607143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c0009550 con 0x7f10c40c2180 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10c9c45700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10b400c4e0 tx=0x7f10b400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40052d0 con 0x7f10c40c2180 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b4005430 con 0x7f10c40c2180 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c414e4d0 con 0x7f10c40c2180 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40038c0 con 0x7f10c40c2180 2024-05-08T23:41:12.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.434+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c41e2350 con 0x7f10c40c2180 2024-05-08T23:41:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.435+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c41e3860 con 0x7f10c40c2180 2024-05-08T23:41:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.436+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f10b4003a20 con 0x7f10c40c2180 2024-05-08T23:41:12.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.437+0000 7f10baffd700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.437+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f10b4088940 con 0x7f10c40c2180 2024-05-08T23:41:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.437+0000 7f10ca446700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.437+0000 7f10ca446700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 0x7f10a806d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009840 tx=0x7f10c001b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.441+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f10b4053b30 con 0x7f10c40c2180 2024-05-08T23:41:12.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.655+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10c4150f10 con 0x7f10a806ac90 2024-05-08T23:41:12.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.665+0000 7f10baffd700 1 -- 172.21.15.33:0/3488607143 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290350 (secure 0 0 0) 0x7f10c000bbf0 con 0x7f10a806ac90 2024-05-08T23:41:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 msgr2=0x7f10a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 0x7f10a806d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009840 tx=0x7f10c001b040 comp rx=0 tx=0).stop 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 msgr2=0x7f10c40c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10b400c4e0 tx=0x7f10b400c7f0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 shutdown_connections 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f10a806ac90 0x7f10a806d150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10c40c8e50 0x7f10c4145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f10c406b660 0x7f10c40cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 --2- 172.21.15.33:0/3488607143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10c40c2180 0x7f10c40c9bb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 >> 172.21.15.33:0/3488607143 conn(0x7f10c40c8360 msgr2=0x7f10c40c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 shutdown_connections 2024-05-08T23:41:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.669+0000 7f10cc6aa700 1 -- 172.21.15.33:0/3488607143 wait complete. 2024-05-08T23:41:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:41:12.681 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-05-08T23:41:11.494814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17883856,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320946,"num_write_kb":49019244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421683,"ondisk_log_size":421683,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4065176,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4028211,"kb_avail":750909544,"statfs":{"total":773094113280,"available":768931373056,"internally_reserved":0,"allocated":37773312,"data_stored":36471936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4124888632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003299"},"pg_stats":[{"pgid":"3.3f","version":"23'5694","reported_seq":5710,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.882557+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:40:50.882557+0000","last_peered":"2024-05-08T23:40:50.882557+0000","last_clean":"2024-05-08T23:40:50.882557+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:40:50.882557+0000","last_undegraded":"2024-05-08T23:40:50.882557+0000","last_fullsized":"2024-05-08T23:40:50.882557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8388","reported_seq":8404,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.810549+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:40:50.810549+0000","last_peered":"2024-05-08T23:40:50.810549+0000","last_clean":"2024-05-08T23:40:50.810549+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:40:50.810549+0000","last_undegraded":"2024-05-08T23:40:50.810549+0000","last_fullsized":"2024-05-08T23:40:50.810549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8388,"log_dups_size":0,"ondisk_log_size":8388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6507","reported_seq":6523,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.805708+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:40:50.805708+0000","last_peered":"2024-05-08T23:40:50.805708+0000","last_clean":"2024-05-08T23:40:50.805708+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:40:50.805708+0000","last_undegraded":"2024-05-08T23:40:50.805708+0000","last_fullsized":"2024-05-08T23:40:50.805708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6507,"log_dups_size":0,"ondisk_log_size":6507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5987","reported_seq":6003,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.725504+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:40:50.725504+0000","last_peered":"2024-05-08T23:40:50.725504+0000","last_clean":"2024-05-08T23:40:50.725504+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:40:50.725504+0000","last_undegraded":"2024-05-08T23:40:50.725504+0000","last_fullsized":"2024-05-08T23:40:50.725504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5987,"log_dups_size":0,"ondisk_log_size":5987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7603","reported_seq":7616,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885580+0000","last_change":"2024-05-08T23:32:59.855454+0000","last_active":"2024-05-08T23:40:50.885580+0000","last_peered":"2024-05-08T23:40:50.885580+0000","last_clean":"2024-05-08T23:40:50.885580+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:40:50.885580+0000","last_undegraded":"2024-05-08T23:40:50.885580+0000","last_fullsized":"2024-05-08T23:40:50.885580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7603,"log_dups_size":0,"ondisk_log_size":7603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:58.324852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5747","reported_seq":5762,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.540996+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:40:50.540996+0000","last_peered":"2024-05-08T23:40:50.540996+0000","last_clean":"2024-05-08T23:40:50.540996+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:40:50.540996+0000","last_undegraded":"2024-05-08T23:40:50.540996+0000","last_fullsized":"2024-05-08T23:40:50.540996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5747,"log_dups_size":0,"ondisk_log_size":5747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6492","reported_seq":6507,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.796565+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:40:50.796565+0000","last_peered":"2024-05-08T23:40:50.796565+0000","last_clean":"2024-05-08T23:40:50.796565+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:40:50.796565+0000","last_undegraded":"2024-05-08T23:40:50.796565+0000","last_fullsized":"2024-05-08T23:40:50.796565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6884","reported_seq":6899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849483+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:40:50.849483+0000","last_peered":"2024-05-08T23:40:50.849483+0000","last_clean":"2024-05-08T23:40:50.849483+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:40:50.849483+0000","last_undegraded":"2024-05-08T23:40:50.849483+0000","last_fullsized":"2024-05-08T23:40:50.849483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6884,"log_dups_size":0,"ondisk_log_size":6884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8081","reported_seq":8096,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.135086+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:40:50.135086+0000","last_peered":"2024-05-08T23:40:50.135086+0000","last_clean":"2024-05-08T23:40:50.135086+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:40:50.135086+0000","last_undegraded":"2024-05-08T23:40:50.135086+0000","last_fullsized":"2024-05-08T23:40:50.135086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7409","reported_seq":7424,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.785232+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:40:50.785232+0000","last_peered":"2024-05-08T23:40:50.785232+0000","last_clean":"2024-05-08T23:40:50.785232+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:40:50.785232+0000","last_undegraded":"2024-05-08T23:40:50.785232+0000","last_fullsized":"2024-05-08T23:40:50.785232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6316","reported_seq":6331,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.655933+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:40:50.655933+0000","last_peered":"2024-05-08T23:40:50.655933+0000","last_clean":"2024-05-08T23:40:50.655933+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:40:50.655933+0000","last_undegraded":"2024-05-08T23:40:50.655933+0000","last_fullsized":"2024-05-08T23:40:50.655933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5502","reported_seq":5517,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.801703+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:40:50.801703+0000","last_peered":"2024-05-08T23:40:50.801703+0000","last_clean":"2024-05-08T23:40:50.801703+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:40:50.801703+0000","last_undegraded":"2024-05-08T23:40:50.801703+0000","last_fullsized":"2024-05-08T23:40:50.801703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6029","reported_seq":6044,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.842402+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:40:50.842402+0000","last_peered":"2024-05-08T23:40:50.842402+0000","last_clean":"2024-05-08T23:40:50.842402+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:40:50.842402+0000","last_undegraded":"2024-05-08T23:40:50.842402+0000","last_fullsized":"2024-05-08T23:40:50.842402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6029,"log_dups_size":0,"ondisk_log_size":6029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6673","reported_seq":6688,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.804806+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:40:50.804806+0000","last_peered":"2024-05-08T23:40:50.804806+0000","last_clean":"2024-05-08T23:40:50.804806+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:40:50.804806+0000","last_undegraded":"2024-05-08T23:40:50.804806+0000","last_fullsized":"2024-05-08T23:40:50.804806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6514","reported_seq":6529,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836662+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:40:50.836662+0000","last_peered":"2024-05-08T23:40:50.836662+0000","last_clean":"2024-05-08T23:40:50.836662+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:40:50.836662+0000","last_undegraded":"2024-05-08T23:40:50.836662+0000","last_fullsized":"2024-05-08T23:40:50.836662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5772","reported_seq":5785,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.818932+0000","last_change":"2024-05-08T23:32:59.856316+0000","last_active":"2024-05-08T23:40:50.818932+0000","last_peered":"2024-05-08T23:40:50.818932+0000","last_clean":"2024-05-08T23:40:50.818932+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:40:50.818932+0000","last_undegraded":"2024-05-08T23:40:50.818932+0000","last_fullsized":"2024-05-08T23:40:50.818932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:19.763425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7973","reported_seq":7988,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.808102+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:40:50.808102+0000","last_peered":"2024-05-08T23:40:50.808102+0000","last_clean":"2024-05-08T23:40:50.808102+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:40:50.808102+0000","last_undegraded":"2024-05-08T23:40:50.808102+0000","last_fullsized":"2024-05-08T23:40:50.808102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7973,"log_dups_size":0,"ondisk_log_size":7973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5300","reported_seq":5315,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901529+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:40:50.901529+0000","last_peered":"2024-05-08T23:40:50.901529+0000","last_clean":"2024-05-08T23:40:50.901529+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:40:50.901529+0000","last_undegraded":"2024-05-08T23:40:50.901529+0000","last_fullsized":"2024-05-08T23:40:50.901529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6323","reported_seq":6338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.766273+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:40:50.766273+0000","last_peered":"2024-05-08T23:40:50.766273+0000","last_clean":"2024-05-08T23:40:50.766273+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:40:50.766273+0000","last_undegraded":"2024-05-08T23:40:50.766273+0000","last_fullsized":"2024-05-08T23:40:50.766273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'383","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.822638+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:40:50.822638+0000","last_peered":"2024-05-08T23:40:50.822638+0000","last_clean":"2024-05-08T23:40:50.822638+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:40:50.822638+0000","last_undegraded":"2024-05-08T23:40:50.822638+0000","last_fullsized":"2024-05-08T23:40:50.822638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6842","reported_seq":6856,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852287+0000","last_change":"2024-05-08T23:33:00.026361+0000","last_active":"2024-05-08T23:40:50.852287+0000","last_peered":"2024-05-08T23:40:50.852287+0000","last_clean":"2024-05-08T23:40:50.852287+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:40:50.852287+0000","last_undegraded":"2024-05-08T23:40:50.852287+0000","last_fullsized":"2024-05-08T23:40:50.852287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:26.768597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828674+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:40:50.828674+0000","last_peered":"2024-05-08T23:40:50.828674+0000","last_clean":"2024-05-08T23:40:50.828674+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:40:50.828674+0000","last_undegraded":"2024-05-08T23:40:50.828674+0000","last_fullsized":"2024-05-08T23:40:50.828674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7911","reported_seq":7935,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.906938+0000","last_change":"2024-05-08T23:32:59.859469+0000","last_active":"2024-05-08T23:40:50.906938+0000","last_peered":"2024-05-08T23:40:50.906938+0000","last_clean":"2024-05-08T23:40:50.906938+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:40:50.906938+0000","last_undegraded":"2024-05-08T23:40:50.906938+0000","last_fullsized":"2024-05-08T23:40:50.906938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:35.269871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'10","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.837769+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:40:50.837769+0000","last_peered":"2024-05-08T23:40:50.837769+0000","last_clean":"2024-05-08T23:40:50.837769+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:40:50.837769+0000","last_undegraded":"2024-05-08T23:40:50.837769+0000","last_fullsized":"2024-05-08T23:40:50.837769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4519","reported_seq":4534,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.644012+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:40:50.644012+0000","last_peered":"2024-05-08T23:40:50.644012+0000","last_clean":"2024-05-08T23:40:50.644012+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:40:50.644012+0000","last_undegraded":"2024-05-08T23:40:50.644012+0000","last_fullsized":"2024-05-08T23:40:50.644012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.715531+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:40:50.715531+0000","last_peered":"2024-05-08T23:40:50.715531+0000","last_clean":"2024-05-08T23:40:50.715531+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:40:50.715531+0000","last_undegraded":"2024-05-08T23:40:50.715531+0000","last_fullsized":"2024-05-08T23:40:50.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7897","reported_seq":7912,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828353+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:40:50.828353+0000","last_peered":"2024-05-08T23:40:50.828353+0000","last_clean":"2024-05-08T23:40:50.828353+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:40:50.828353+0000","last_undegraded":"2024-05-08T23:40:50.828353+0000","last_fullsized":"2024-05-08T23:40:50.828353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1096","reported_seq":1112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.825119+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:40:50.825119+0000","last_peered":"2024-05-08T23:40:50.825119+0000","last_clean":"2024-05-08T23:40:50.825119+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:40:50.825119+0000","last_undegraded":"2024-05-08T23:40:50.825119+0000","last_fullsized":"2024-05-08T23:40:50.825119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5444","reported_seq":5459,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.736987+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:40:50.736987+0000","last_peered":"2024-05-08T23:40:50.736987+0000","last_clean":"2024-05-08T23:40:50.736987+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:40:50.736987+0000","last_undegraded":"2024-05-08T23:40:50.736987+0000","last_fullsized":"2024-05-08T23:40:50.736987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5444,"log_dups_size":0,"ondisk_log_size":5444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'25","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.878177+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:40:50.878177+0000","last_peered":"2024-05-08T23:40:50.878177+0000","last_clean":"2024-05-08T23:40:50.878177+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:40:50.878177+0000","last_undegraded":"2024-05-08T23:40:50.878177+0000","last_fullsized":"2024-05-08T23:40:50.878177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5057","reported_seq":5072,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.807049+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:40:50.807049+0000","last_peered":"2024-05-08T23:40:50.807049+0000","last_clean":"2024-05-08T23:40:50.807049+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:40:50.807049+0000","last_undegraded":"2024-05-08T23:40:50.807049+0000","last_fullsized":"2024-05-08T23:40:50.807049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836686+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:40:50.836686+0000","last_peered":"2024-05-08T23:40:50.836686+0000","last_clean":"2024-05-08T23:40:50.836686+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:40:50.836686+0000","last_undegraded":"2024-05-08T23:40:50.836686+0000","last_fullsized":"2024-05-08T23:40:50.836686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6593","reported_seq":6606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901627+0000","last_change":"2024-05-08T23:32:59.856036+0000","last_active":"2024-05-08T23:40:50.901627+0000","last_peered":"2024-05-08T23:40:50.901627+0000","last_clean":"2024-05-08T23:40:50.901627+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:40:50.901627+0000","last_undegraded":"2024-05-08T23:40:50.901627+0000","last_fullsized":"2024-05-08T23:40:50.901627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6593,"log_dups_size":0,"ondisk_log_size":6593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:07.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:37:06.070170+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:37:06.070170+0000","last_peered":"2024-05-08T23:37:06.070170+0000","last_clean":"2024-05-08T23:37:06.070170+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:37:06.070170+0000","last_undegraded":"2024-05-08T23:37:06.070170+0000","last_fullsized":"2024-05-08T23:37:06.070170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7580","reported_seq":7595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.855374+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:40:50.855374+0000","last_peered":"2024-05-08T23:40:50.855374+0000","last_clean":"2024-05-08T23:40:50.855374+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:40:50.855374+0000","last_undegraded":"2024-05-08T23:40:50.855374+0000","last_fullsized":"2024-05-08T23:40:50.855374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7580,"log_dups_size":0,"ondisk_log_size":7580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.812887+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:40:50.812887+0000","last_peered":"2024-05-08T23:40:50.812887+0000","last_clean":"2024-05-08T23:40:50.812887+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:40:50.812887+0000","last_undegraded":"2024-05-08T23:40:50.812887+0000","last_fullsized":"2024-05-08T23:40:50.812887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6493","reported_seq":6508,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885950+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:40:50.885950+0000","last_peered":"2024-05-08T23:40:50.885950+0000","last_clean":"2024-05-08T23:40:50.885950+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:40:50.885950+0000","last_undegraded":"2024-05-08T23:40:50.885950+0000","last_fullsized":"2024-05-08T23:40:50.885950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6493,"log_dups_size":0,"ondisk_log_size":6493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.804197+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:40:50.804197+0000","last_peered":"2024-05-08T23:40:50.804197+0000","last_clean":"2024-05-08T23:40:50.804197+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:40:50.804197+0000","last_undegraded":"2024-05-08T23:40:50.804197+0000","last_fullsized":"2024-05-08T23:40:50.804197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5812","reported_seq":5835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.807383+0000","last_change":"2024-05-08T23:32:59.859486+0000","last_active":"2024-05-08T23:40:50.807383+0000","last_peered":"2024-05-08T23:40:50.807383+0000","last_clean":"2024-05-08T23:40:50.807383+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:40:50.807383+0000","last_undegraded":"2024-05-08T23:40:50.807383+0000","last_fullsized":"2024-05-08T23:40:50.807383+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:23.076314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'17","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869954+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:40:50.869954+0000","last_peered":"2024-05-08T23:40:50.869954+0000","last_clean":"2024-05-08T23:40:50.869954+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:40:50.869954+0000","last_undegraded":"2024-05-08T23:40:50.869954+0000","last_fullsized":"2024-05-08T23:40:50.869954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8441","reported_seq":8456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885980+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:40:50.885980+0000","last_peered":"2024-05-08T23:40:50.885980+0000","last_clean":"2024-05-08T23:40:50.885980+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:40:50.885980+0000","last_undegraded":"2024-05-08T23:40:50.885980+0000","last_fullsized":"2024-05-08T23:40:50.885980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8441,"log_dups_size":0,"ondisk_log_size":8441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'222","reported_seq":238,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.718826+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:40:50.718826+0000","last_peered":"2024-05-08T23:40:50.718826+0000","last_clean":"2024-05-08T23:40:50.718826+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:40:50.718826+0000","last_undegraded":"2024-05-08T23:40:50.718826+0000","last_fullsized":"2024-05-08T23:40:50.718826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6150","reported_seq":6165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.633129+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:40:50.633129+0000","last_peered":"2024-05-08T23:40:50.633129+0000","last_clean":"2024-05-08T23:40:50.633129+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:40:50.633129+0000","last_undegraded":"2024-05-08T23:40:50.633129+0000","last_fullsized":"2024-05-08T23:40:50.633129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6150,"log_dups_size":0,"ondisk_log_size":6150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854210+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:40:50.854210+0000","last_peered":"2024-05-08T23:40:50.854210+0000","last_clean":"2024-05-08T23:40:50.854210+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:40:50.854210+0000","last_undegraded":"2024-05-08T23:40:50.854210+0000","last_fullsized":"2024-05-08T23:40:50.854210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5975","reported_seq":5990,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.722519+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:40:50.722519+0000","last_peered":"2024-05-08T23:40:50.722519+0000","last_clean":"2024-05-08T23:40:50.722519+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:40:50.722519+0000","last_undegraded":"2024-05-08T23:40:50.722519+0000","last_fullsized":"2024-05-08T23:40:50.722519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5975,"log_dups_size":0,"ondisk_log_size":5975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.883912+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:40:50.883912+0000","last_peered":"2024-05-08T23:40:50.883912+0000","last_clean":"2024-05-08T23:40:50.883912+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:40:50.883912+0000","last_undegraded":"2024-05-08T23:40:50.883912+0000","last_fullsized":"2024-05-08T23:40:50.883912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7310","reported_seq":7325,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.859323+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:40:50.859323+0000","last_peered":"2024-05-08T23:40:50.859323+0000","last_clean":"2024-05-08T23:40:50.859323+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:40:50.859323+0000","last_undegraded":"2024-05-08T23:40:50.859323+0000","last_fullsized":"2024-05-08T23:40:50.859323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7310,"log_dups_size":0,"ondisk_log_size":7310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.831450+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:40:50.831450+0000","last_peered":"2024-05-08T23:40:50.831450+0000","last_clean":"2024-05-08T23:40:50.831450+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:40:50.831450+0000","last_undegraded":"2024-05-08T23:40:50.831450+0000","last_fullsized":"2024-05-08T23:40:50.831450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7011","reported_seq":7026,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.714630+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:40:50.714630+0000","last_peered":"2024-05-08T23:40:50.714630+0000","last_clean":"2024-05-08T23:40:50.714630+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:40:50.714630+0000","last_undegraded":"2024-05-08T23:40:50.714630+0000","last_fullsized":"2024-05-08T23:40:50.714630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'15","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828797+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:40:50.828797+0000","last_peered":"2024-05-08T23:40:50.828797+0000","last_clean":"2024-05-08T23:40:50.828797+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:40:50.828797+0000","last_undegraded":"2024-05-08T23:40:50.828797+0000","last_fullsized":"2024-05-08T23:40:50.828797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6605","reported_seq":6620,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.792042+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:40:50.792042+0000","last_peered":"2024-05-08T23:40:50.792042+0000","last_clean":"2024-05-08T23:40:50.792042+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:40:50.792042+0000","last_undegraded":"2024-05-08T23:40:50.792042+0000","last_fullsized":"2024-05-08T23:40:50.792042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836668+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:40:50.836668+0000","last_peered":"2024-05-08T23:40:50.836668+0000","last_clean":"2024-05-08T23:40:50.836668+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:40:50.836668+0000","last_undegraded":"2024-05-08T23:40:50.836668+0000","last_fullsized":"2024-05-08T23:40:50.836668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6071","reported_seq":6094,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870424+0000","last_change":"2024-05-08T23:33:00.026523+0000","last_active":"2024-05-08T23:40:50.870424+0000","last_peered":"2024-05-08T23:40:50.870424+0000","last_clean":"2024-05-08T23:40:50.870424+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:40:50.870424+0000","last_undegraded":"2024-05-08T23:40:50.870424+0000","last_fullsized":"2024-05-08T23:40:50.870424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6071,"log_dups_size":0,"ondisk_log_size":6071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:23.920007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826077+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:40:50.826077+0000","last_peered":"2024-05-08T23:40:50.826077+0000","last_clean":"2024-05-08T23:40:50.826077+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:40:50.826077+0000","last_undegraded":"2024-05-08T23:40:50.826077+0000","last_fullsized":"2024-05-08T23:40:50.826077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5809","reported_seq":5825,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854937+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:40:50.854937+0000","last_peered":"2024-05-08T23:40:50.854937+0000","last_clean":"2024-05-08T23:40:50.854937+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:40:50.854937+0000","last_undegraded":"2024-05-08T23:40:50.854937+0000","last_fullsized":"2024-05-08T23:40:50.854937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:27.185535+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:38:27.185535+0000","last_peered":"2024-05-08T23:38:27.185535+0000","last_clean":"2024-05-08T23:38:27.185535+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:38:27.185535+0000","last_undegraded":"2024-05-08T23:38:27.185535+0000","last_fullsized":"2024-05-08T23:38:27.185535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6329","reported_seq":6345,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.684184+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:40:50.684184+0000","last_peered":"2024-05-08T23:40:50.684184+0000","last_clean":"2024-05-08T23:40:50.684184+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:40:50.684184+0000","last_undegraded":"2024-05-08T23:40:50.684184+0000","last_fullsized":"2024-05-08T23:40:50.684184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6273","reported_seq":6289,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.844273+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:40:50.844273+0000","last_peered":"2024-05-08T23:40:50.844273+0000","last_clean":"2024-05-08T23:40:50.844273+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:40:50.844273+0000","last_undegraded":"2024-05-08T23:40:50.844273+0000","last_fullsized":"2024-05-08T23:40:50.844273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6273,"log_dups_size":0,"ondisk_log_size":6273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'27","reported_seq":68,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828876+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:40:50.828876+0000","last_peered":"2024-05-08T23:40:50.828876+0000","last_clean":"2024-05-08T23:40:50.828876+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:40:50.828876+0000","last_undegraded":"2024-05-08T23:40:50.828876+0000","last_fullsized":"2024-05-08T23:40:50.828876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6847","reported_seq":6863,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.265524+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:40:50.265524+0000","last_peered":"2024-05-08T23:40:50.265524+0000","last_clean":"2024-05-08T23:40:50.265524+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:40:50.265524+0000","last_undegraded":"2024-05-08T23:40:50.265524+0000","last_fullsized":"2024-05-08T23:40:50.265524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.805976+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:40:50.805976+0000","last_peered":"2024-05-08T23:40:50.805976+0000","last_clean":"2024-05-08T23:40:50.805976+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:40:50.805976+0000","last_undegraded":"2024-05-08T23:40:50.805976+0000","last_fullsized":"2024-05-08T23:40:50.805976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:16.309946+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:38:16.309946+0000","last_peered":"2024-05-08T23:38:16.309946+0000","last_clean":"2024-05-08T23:38:16.309946+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:38:16.309946+0000","last_undegraded":"2024-05-08T23:38:16.309946+0000","last_fullsized":"2024-05-08T23:38:16.309946+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6391","reported_seq":6407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.792470+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:40:50.792470+0000","last_peered":"2024-05-08T23:40:50.792470+0000","last_clean":"2024-05-08T23:40:50.792470+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:40:50.792470+0000","last_undegraded":"2024-05-08T23:40:50.792470+0000","last_fullsized":"2024-05-08T23:40:50.792470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6391,"log_dups_size":0,"ondisk_log_size":6391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849138+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:40:50.849138+0000","last_peered":"2024-05-08T23:40:50.849138+0000","last_clean":"2024-05-08T23:40:50.849138+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:40:50.849138+0000","last_undegraded":"2024-05-08T23:40:50.849138+0000","last_fullsized":"2024-05-08T23:40:50.849138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:38.734890+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:40:38.734890+0000","last_peered":"2024-05-08T23:40:38.734890+0000","last_clean":"2024-05-08T23:40:38.734890+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:40:38.734890+0000","last_undegraded":"2024-05-08T23:40:38.734890+0000","last_fullsized":"2024-05-08T23:40:38.734890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7006","reported_seq":7022,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.821144+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:40:50.821144+0000","last_peered":"2024-05-08T23:40:50.821144+0000","last_clean":"2024-05-08T23:40:50.821144+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:40:50.821144+0000","last_undegraded":"2024-05-08T23:40:50.821144+0000","last_fullsized":"2024-05-08T23:40:50.821144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7006,"log_dups_size":0,"ondisk_log_size":7006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'86","reported_seq":103,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826703+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:40:50.826703+0000","last_peered":"2024-05-08T23:40:50.826703+0000","last_clean":"2024-05-08T23:40:50.826703+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:40:50.826703+0000","last_undegraded":"2024-05-08T23:40:50.826703+0000","last_fullsized":"2024-05-08T23:40:50.826703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6426","reported_seq":6442,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.839221+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:40:50.839221+0000","last_peered":"2024-05-08T23:40:50.839221+0000","last_clean":"2024-05-08T23:40:50.839221+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:40:50.839221+0000","last_undegraded":"2024-05-08T23:40:50.839221+0000","last_fullsized":"2024-05-08T23:40:50.839221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6426,"log_dups_size":0,"ondisk_log_size":6426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.423772+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:40:51.423772+0000","last_peered":"2024-05-08T23:40:51.423772+0000","last_clean":"2024-05-08T23:40:51.423772+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:40:51.423772+0000","last_undegraded":"2024-05-08T23:40:51.423772+0000","last_fullsized":"2024-05-08T23:40:51.423772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5451","reported_seq":5467,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.795395+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:40:50.795395+0000","last_peered":"2024-05-08T23:40:50.795395+0000","last_clean":"2024-05-08T23:40:50.795395+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:40:50.795395+0000","last_undegraded":"2024-05-08T23:40:50.795395+0000","last_fullsized":"2024-05-08T23:40:50.795395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.851008+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:40:50.851008+0000","last_peered":"2024-05-08T23:40:50.851008+0000","last_clean":"2024-05-08T23:40:50.851008+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:40:50.851008+0000","last_undegraded":"2024-05-08T23:40:50.851008+0000","last_fullsized":"2024-05-08T23:40:50.851008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5843","reported_seq":5859,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870876+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:40:50.870876+0000","last_peered":"2024-05-08T23:40:50.870876+0000","last_clean":"2024-05-08T23:40:50.870876+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:40:50.870876+0000","last_undegraded":"2024-05-08T23:40:50.870876+0000","last_fullsized":"2024-05-08T23:40:50.870876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5843,"log_dups_size":0,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.835933+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:40:50.835933+0000","last_peered":"2024-05-08T23:40:50.835933+0000","last_clean":"2024-05-08T23:40:50.835933+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:40:50.835933+0000","last_undegraded":"2024-05-08T23:40:50.835933+0000","last_fullsized":"2024-05-08T23:40:50.835933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6479","reported_seq":6495,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.624049+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:40:50.624049+0000","last_peered":"2024-05-08T23:40:50.624049+0000","last_clean":"2024-05-08T23:40:50.624049+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:40:50.624049+0000","last_undegraded":"2024-05-08T23:40:50.624049+0000","last_fullsized":"2024-05-08T23:40:50.624049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6479,"log_dups_size":0,"ondisk_log_size":6479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.875794+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:40:50.875794+0000","last_peered":"2024-05-08T23:40:50.875794+0000","last_clean":"2024-05-08T23:40:50.875794+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:40:50.875794+0000","last_undegraded":"2024-05-08T23:40:50.875794+0000","last_fullsized":"2024-05-08T23:40:50.875794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5333","reported_seq":5348,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.796682+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:40:50.796682+0000","last_peered":"2024-05-08T23:40:50.796682+0000","last_clean":"2024-05-08T23:40:50.796682+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:40:50.796682+0000","last_undegraded":"2024-05-08T23:40:50.796682+0000","last_fullsized":"2024-05-08T23:40:50.796682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5333,"log_dups_size":0,"ondisk_log_size":5333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'685","reported_seq":701,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852470+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:40:50.852470+0000","last_peered":"2024-05-08T23:40:50.852470+0000","last_clean":"2024-05-08T23:40:50.852470+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:40:50.852470+0000","last_undegraded":"2024-05-08T23:40:50.852470+0000","last_fullsized":"2024-05-08T23:40:50.852470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5933","reported_seq":5949,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.871976+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:40:50.871976+0000","last_peered":"2024-05-08T23:40:50.871976+0000","last_clean":"2024-05-08T23:40:50.871976+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:40:50.871976+0000","last_undegraded":"2024-05-08T23:40:50.871976+0000","last_fullsized":"2024-05-08T23:40:50.871976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5933,"log_dups_size":0,"ondisk_log_size":5933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.896708+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:40:50.896708+0000","last_peered":"2024-05-08T23:40:50.896708+0000","last_clean":"2024-05-08T23:40:50.896708+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:40:50.896708+0000","last_undegraded":"2024-05-08T23:40:50.896708+0000","last_fullsized":"2024-05-08T23:40:50.896708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6537","reported_seq":6552,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.899748+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:40:50.899748+0000","last_peered":"2024-05-08T23:40:50.899748+0000","last_clean":"2024-05-08T23:40:50.899748+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:40:50.899748+0000","last_undegraded":"2024-05-08T23:40:50.899748+0000","last_fullsized":"2024-05-08T23:40:50.899748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6537,"log_dups_size":0,"ondisk_log_size":6537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:39:01.910099+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:39:01.910099+0000","last_peered":"2024-05-08T23:39:01.910099+0000","last_clean":"2024-05-08T23:39:01.910099+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:39:01.910099+0000","last_undegraded":"2024-05-08T23:39:01.910099+0000","last_fullsized":"2024-05-08T23:39:01.910099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6818","reported_seq":6833,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.897583+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:40:50.897583+0000","last_peered":"2024-05-08T23:40:50.897583+0000","last_clean":"2024-05-08T23:40:50.897583+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:40:50.897583+0000","last_undegraded":"2024-05-08T23:40:50.897583+0000","last_fullsized":"2024-05-08T23:40:50.897583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6818,"log_dups_size":0,"ondisk_log_size":6818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828990+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:40:50.828990+0000","last_peered":"2024-05-08T23:40:50.828990+0000","last_clean":"2024-05-08T23:40:50.828990+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:40:50.828990+0000","last_undegraded":"2024-05-08T23:40:50.828990+0000","last_fullsized":"2024-05-08T23:40:50.828990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7691","reported_seq":7706,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.671715+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:40:50.671715+0000","last_peered":"2024-05-08T23:40:50.671715+0000","last_clean":"2024-05-08T23:40:50.671715+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:40:50.671715+0000","last_undegraded":"2024-05-08T23:40:50.671715+0000","last_fullsized":"2024-05-08T23:40:50.671715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7797","reported_seq":7813,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.827905+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:40:50.827905+0000","last_peered":"2024-05-08T23:40:50.827905+0000","last_clean":"2024-05-08T23:40:50.827905+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:40:50.827905+0000","last_undegraded":"2024-05-08T23:40:50.827905+0000","last_fullsized":"2024-05-08T23:40:50.827905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7797,"log_dups_size":0,"ondisk_log_size":7797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6562","reported_seq":6578,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.602602+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:40:50.602602+0000","last_peered":"2024-05-08T23:40:50.602602+0000","last_clean":"2024-05-08T23:40:50.602602+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:40:50.602602+0000","last_undegraded":"2024-05-08T23:40:50.602602+0000","last_fullsized":"2024-05-08T23:40:50.602602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8077","reported_seq":8093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.884270+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:40:50.884270+0000","last_peered":"2024-05-08T23:40:50.884270+0000","last_clean":"2024-05-08T23:40:50.884270+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:40:50.884270+0000","last_undegraded":"2024-05-08T23:40:50.884270+0000","last_fullsized":"2024-05-08T23:40:50.884270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5650","reported_seq":5666,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.745990+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:40:50.745990+0000","last_peered":"2024-05-08T23:40:50.745990+0000","last_clean":"2024-05-08T23:40:50.745990+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:40:50.745990+0000","last_undegraded":"2024-05-08T23:40:50.745990+0000","last_fullsized":"2024-05-08T23:40:50.745990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5650,"log_dups_size":0,"ondisk_log_size":5650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6013","reported_seq":6029,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.789472+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:40:50.789472+0000","last_peered":"2024-05-08T23:40:50.789472+0000","last_clean":"2024-05-08T23:40:50.789472+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:40:50.789472+0000","last_undegraded":"2024-05-08T23:40:50.789472+0000","last_fullsized":"2024-05-08T23:40:50.789472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7047","reported_seq":7063,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.900550+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:40:50.900550+0000","last_peered":"2024-05-08T23:40:50.900550+0000","last_clean":"2024-05-08T23:40:50.900550+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:40:50.900550+0000","last_undegraded":"2024-05-08T23:40:50.900550+0000","last_fullsized":"2024-05-08T23:40:50.900550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5651","reported_seq":5667,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.880110+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:40:50.880110+0000","last_peered":"2024-05-08T23:40:50.880110+0000","last_clean":"2024-05-08T23:40:50.880110+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:40:50.880110+0000","last_undegraded":"2024-05-08T23:40:50.880110+0000","last_fullsized":"2024-05-08T23:40:50.880110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5651,"log_dups_size":0,"ondisk_log_size":5651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7696","reported_seq":7712,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.880040+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:40:50.880040+0000","last_peered":"2024-05-08T23:40:50.880040+0000","last_clean":"2024-05-08T23:40:50.880040+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:40:50.880040+0000","last_undegraded":"2024-05-08T23:40:50.880040+0000","last_fullsized":"2024-05-08T23:40:50.880040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7696,"log_dups_size":0,"ondisk_log_size":7696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5551","reported_seq":5567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869705+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:40:50.869705+0000","last_peered":"2024-05-08T23:40:50.869705+0000","last_clean":"2024-05-08T23:40:50.869705+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:40:50.869705+0000","last_undegraded":"2024-05-08T23:40:50.869705+0000","last_fullsized":"2024-05-08T23:40:50.869705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5551,"log_dups_size":0,"ondisk_log_size":5551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6385","reported_seq":6400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836597+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:40:50.836597+0000","last_peered":"2024-05-08T23:40:50.836597+0000","last_clean":"2024-05-08T23:40:50.836597+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:40:50.836597+0000","last_undegraded":"2024-05-08T23:40:50.836597+0000","last_fullsized":"2024-05-08T23:40:50.836597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6385,"log_dups_size":0,"ondisk_log_size":6385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7231","reported_seq":7246,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.609971+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:40:50.609971+0000","last_peered":"2024-05-08T23:40:50.609971+0000","last_clean":"2024-05-08T23:40:50.609971+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:40:50.609971+0000","last_undegraded":"2024-05-08T23:40:50.609971+0000","last_fullsized":"2024-05-08T23:40:50.609971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7231,"log_dups_size":0,"ondisk_log_size":7231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6799","reported_seq":6814,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.884006+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:40:50.884006+0000","last_peered":"2024-05-08T23:40:50.884006+0000","last_clean":"2024-05-08T23:40:50.884006+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:40:50.884006+0000","last_undegraded":"2024-05-08T23:40:50.884006+0000","last_fullsized":"2024-05-08T23:40:50.884006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6386","reported_seq":6401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.814908+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:40:50.814908+0000","last_peered":"2024-05-08T23:40:50.814908+0000","last_clean":"2024-05-08T23:40:50.814908+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:40:50.814908+0000","last_undegraded":"2024-05-08T23:40:50.814908+0000","last_fullsized":"2024-05-08T23:40:50.814908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":418916,"ondisk_log_size":418916,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293488,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2678,"num_write_kb":19121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34586976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2650,"ondisk_log_size":2650,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633788,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":620670,"kb_avail":93738052,"statfs":{"total":96636764160,"available":95987765248,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":635566539},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":8.4469999999999992,"15min":3.2210000000000001},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":45.125999999999998,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":9.0259999999999998,"5min":7.827,"15min":3.0419999999999998},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":37.293999999999997,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.194,"5min":4.4950000000000001,"15min":1.9470000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":13.531000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":9.1940000000000008,"5min":7.9210000000000003,"15min":3.1160000000000001},"min":{"1min":0.40000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":24.387,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.5529999999999999,"5min":4.2610000000000001,"15min":1.8879999999999999},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":15.938000000000001,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":6.0819999999999999,"5min":5.0229999999999997,"15min":2.1219999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":19.565000000000001,"5min":26.084,"15min":26.084},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":6.0049999999999999,"5min":7.1989999999999998,"15min":2.8839999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":19.614000000000001,"5min":52.869,"15min":52.869},"last":0.52900000000000003},{"interface":"front","average":{"1min":10.468,"5min":7.2720000000000002,"15min":2.855},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35499999999999998},"max":{"1min":49.421999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.67500000000000004}]},{"osd":4,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":5.0599999999999996,"15min":2.0430000000000001},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.9139999999999997,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":5.3949999999999996,"5min":3.8919999999999999,"15min":1.6950000000000001},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":22.901,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":7.1719999999999997,"5min":5.1349999999999998,"15min":2.0910000000000002},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":18.988,"5min":63.732999999999997,"15min":63.732999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":9.2859999999999996,"5min":7.7160000000000002,"15min":2.9860000000000002},"min":{"1min":0.69199999999999995,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":18.504999999999999,"5min":62.558,"15min":62.558},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":10.367000000000001,"5min":7.5259999999999998,"15min":2.9529999999999998},"min":{"1min":0.71799999999999997,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":45.826999999999998,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":5.8150000000000004,"5min":4.6600000000000001,"15min":1.982},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40600000000000003},"max":{"1min":38.801000000000002,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.60499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4709827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.7320000000000002,"5min":6.8529999999999998,"15min":2.7749999999999999},"min":{"1min":0.52500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":19.369,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":6.4900000000000002,"5min":7.8170000000000002,"15min":3.1259999999999999},"min":{"1min":0.47199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":18.379000000000001,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":6.8890000000000002,"5min":6.3970000000000002,"15min":2.8570000000000002},"min":{"1min":0.876,"5min":0.313,"15min":0.313},"max":{"1min":24.693000000000001,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":5.6459999999999999,"5min":5.7210000000000001,"15min":2.6230000000000002},"min":{"1min":0.57899999999999996,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":24.187999999999999,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":4.306,"5min":4.7060000000000004,"15min":2.1030000000000002},"min":{"1min":0.58499999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":15.589,"5min":41.445,"15min":41.445},"last":0.69799999999999995},{"interface":"front","average":{"1min":4.3399999999999999,"5min":4.6550000000000002,"15min":2.0630000000000002},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":15.624000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.77400000000000002}]},{"osd":3,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.4909999999999997,"5min":5.2729999999999997,"15min":2.266},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":15.467000000000001,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":5.8230000000000004,"5min":7.4550000000000001,"15min":3.1749999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":28.600000000000001,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":3.141,"15min":1.446},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.891,"5min":20.913,"15min":20.913},"last":0.5},{"interface":"front","average":{"1min":2.1909999999999998,"5min":3.875,"15min":1.8129999999999999},"min":{"1min":0.52700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.9569999999999999,"5min":35.975999999999999,"15min":35.975999999999999},"last":0.752}]},{"osd":5,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":7.3280000000000003,"5min":6.5369999999999999,"15min":2.657},"min":{"1min":0.64500000000000002,"5min":0.39200000000000002,"15min":0.33100000000000002},"max":{"1min":36.756,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.624},{"interface":"front","average":{"1min":6.0759999999999996,"5min":4.633,"15min":2.0230000000000001},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":23.105,"5min":23.105,"15min":23.105},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":4.2359999999999998,"15min":2.0259999999999998},"min":{"1min":0.44800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":6.9900000000000002,"5min":24.16,"15min":33.463000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":5.7530000000000001,"5min":5.6429999999999998,"15min":2.3460000000000001},"min":{"1min":0.83299999999999996,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.056999999999999,"5min":45.805,"15min":45.805},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":7.673,"5min":6.6539999999999999,"15min":2.5219999999999998},"min":{"1min":0.437,"5min":0.29099999999999998,"15min":0.23400000000000001},"max":{"1min":30.721,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":4.6159999999999997,"5min":5.1420000000000003,"15min":2.1499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":13.93,"5min":48.844999999999999,"15min":48.844999999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.6609999999999996,"5min":4.5510000000000002,"15min":1.897},"min":{"1min":0.32400000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":22.721,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.5819999999999999,"15min":2.3359999999999999},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":25.683,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":7.8360000000000003,"15min":2.9780000000000002},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":21.975000000000001,"5min":52.712000000000003,"15min":52.712000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":4.9900000000000002,"5min":9.1709999999999994,"15min":3.4780000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.32100000000000001},"max":{"1min":25.724,"5min":61.832000000000001,"15min":61.832000000000001},"last":0.70399999999999996}]},{"osd":4,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":6.2939999999999996,"5min":6.4749999999999996,"15min":2.6850000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":23.529,"5min":45.267000000000003,"15min":45.267000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":7.2350000000000003,"5min":6.3070000000000004,"15min":2.5979999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":31.245999999999999,"5min":43.353999999999999,"15min":43.353999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":5.3419999999999996,"5min":6.3719999999999999,"15min":2.6850000000000001},"min":{"1min":0.66300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":16.907,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":6.984,"5min":8.1400000000000006,"15min":3.2719999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":36.280999999999999,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.4470000000000001,"5min":7.5030000000000001,"15min":3.0939999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":11.292999999999999,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":6.8490000000000002,"5min":7.016,"15min":2.8900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":25.600999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":10.384,"5min":8.8420000000000005,"15min":3.4609999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":72.088999999999999,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":10.420999999999999,"5min":9.4009999999999998,"15min":3.7349999999999999},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":72.150000000000006,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.628}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.324,"5min":6.6020000000000003,"15min":2.8780000000000001},"min":{"1min":0.28599999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":47.241999999999997,"5min":47.241999999999997,"15min":101.899},"last":0.47199999999999998},{"interface":"front","average":{"1min":7.3029999999999999,"5min":5.3879999999999999,"15min":2.5339999999999998},"min":{"1min":0.65000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":22.170999999999999,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.44}]},{"osd":2,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":14.221,"5min":8.1920000000000002,"15min":3.5609999999999999},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":134.16300000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":10.577999999999999,"5min":7.7560000000000002,"15min":3.3380000000000001},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":47.277000000000001,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.8740000000000006,"5min":7.0720000000000001,"15min":3.1829999999999998},"min":{"1min":0.59399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":59.796999999999997,"5min":59.796999999999997,"15min":101.929},"last":0.52100000000000002},{"interface":"front","average":{"1min":4.3179999999999996,"5min":4.8780000000000001,"15min":2.2130000000000001},"min":{"1min":0.47499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.606,"5min":95.200999999999993,"15min":95.200999999999993},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.207000000000001,"5min":6.8620000000000001,"15min":3.206},"min":{"1min":0.79500000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":47.194000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.70499999999999996},{"interface":"front","average":{"1min":4.2290000000000001,"5min":5.282,"15min":2.3500000000000001},"min":{"1min":0.54300000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":12.694000000000001,"5min":39.813000000000002,"15min":44.545999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.326000000000001,"5min":6.3300000000000001,"15min":2.8500000000000001},"min":{"1min":1.111,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":38.494,"5min":38.494,"15min":78.780000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":5.0940000000000003,"5min":6.5419999999999998,"15min":2.7690000000000001},"min":{"1min":0.79800000000000004,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":12.657999999999999,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.0920000000000005,"5min":7.1189999999999998,"15min":3.2959999999999998},"min":{"1min":0.85299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":45.744999999999997,"5min":74.013000000000005,"15min":101.869},"last":0.66000000000000003},{"interface":"front","average":{"1min":10.259,"5min":6.2560000000000002,"15min":2.903},"min":{"1min":0.46100000000000002,"5min":0.317,"15min":0.317},"max":{"1min":38.545000000000002,"5min":71.323999999999998,"15min":78.647999999999996},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.394,"5min":6,"15min":2.8479999999999999},"min":{"1min":0.67200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":37.725000000000001,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.80600000000000005},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.9059999999999997,"15min":2.2759999999999998},"min":{"1min":0.61199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":13.746,"5min":27.056000000000001,"15min":44.713999999999999},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":5.6520000000000001,"5min":5.6859999999999999,"15min":2.1960000000000002},"min":{"1min":0.35499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":44.238999999999997,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":6.6989999999999998,"5min":5.024,"15min":2.0939999999999999},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.27700000000000002},"max":{"1min":44.353000000000002,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":10.422000000000001,"5min":6.5999999999999996,"15min":2.5630000000000002},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.255},"max":{"1min":79.540999999999997,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":11.664,"5min":6.3029999999999999,"15min":2.476},"min":{"1min":0.52100000000000002,"5min":0.374,"15min":0.318},"max":{"1min":79.584000000000003,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":8.9100000000000001,"5min":9.1769999999999996,"15min":3.3730000000000002},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":26.959,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":3.363,"5min":5.0359999999999996,"15min":2.0699999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":18.030999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.4589999999999996,"5min":5.0419999999999998,"15min":2.169},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":41.488999999999997,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":4.4950000000000001,"5min":4.5179999999999998,"15min":1.9850000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":17.946999999999999,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.4720000000000004,"5min":6.577,"15min":2.7519999999999998},"min":{"1min":0.88700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":24.843,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":5.4720000000000004,"5min":5.3280000000000003,"15min":2.2970000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":14.734999999999999,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.2080000000000002,"5min":5.0599999999999996,"15min":2.1970000000000001},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":41.396999999999998,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":7.5640000000000001,"5min":6.2050000000000001,"15min":2.5430000000000001},"min":{"1min":0.93999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":24.686,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.2050000000000001,"5min":5.2779999999999996,"15min":2.3100000000000001},"min":{"1min":0.70299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":24.623999999999999,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":4.6799999999999997,"5min":7.0670000000000002,"15min":2.8570000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":17.905000000000001,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":4.125,"5min":6.7089999999999996,"15min":2.5619999999999998},"min":{"1min":0.35299999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":18.997,"5min":67.305999999999997,"15min":67.305999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":6.2910000000000004,"5min":9.1940000000000008,"15min":3.3969999999999998},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.149999999999999,"5min":106.934,"15min":106.934},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":9.6839999999999993,"5min":7.0439999999999996,"15min":2.6669999999999998},"min":{"1min":0.40699999999999997,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":76.628,"5min":76.628,"15min":76.628},"last":0.35299999999999998},{"interface":"front","average":{"1min":4.9690000000000003,"5min":6.4690000000000003,"15min":2.5129999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":19.045999999999999,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":8.5600000000000005,"5min":7.0899999999999999,"15min":3.008},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":57.378,"5min":57.378,"15min":57.378},"last":0.36699999999999999},{"interface":"front","average":{"1min":4.3460000000000001,"5min":5.8959999999999999,"15min":2.363},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":19.800000000000001,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":11.138999999999999,"5min":8.3179999999999996,"15min":3.2629999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":57.313000000000002,"5min":57.313000000000002,"15min":57.313000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":11.608000000000001,"5min":7.4400000000000004,"15min":2.9350000000000001},"min":{"1min":1.5469999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":53.343000000000004,"5min":68.238,"15min":68.238},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.965,"5min":6.7839999999999998,"15min":2.722},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":53.381,"5min":53.381,"15min":53.381},"last":0.79400000000000004},{"interface":"front","average":{"1min":6.71,"5min":7.867,"15min":3.1309999999999998},"min":{"1min":0.47099999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":19.084,"5min":36.213999999999999,"15min":36.213999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.84,"5min":7.8650000000000002,"15min":3.0960000000000001},"min":{"1min":1.2270000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":57.411000000000001,"5min":57.411000000000001,"15min":57.411000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":6.2850000000000001,"5min":5.6790000000000003,"15min":2.3849999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":19.113,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":9.2200000000000006,"15min":3.5830000000000002},"min":{"1min":0.59399999999999997,"5min":0.375,"15min":0.375},"max":{"1min":63.896999999999998,"5min":63.896999999999998,"15min":63.896999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":9.6880000000000006,"5min":7.1580000000000004,"15min":2.9129999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":53.182000000000002,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.75600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9930000000000003,"5min":7.0389999999999997,"15min":2.7829999999999999},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":23.896000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":6.0339999999999998,"5min":7.4790000000000001,"15min":2.956},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":23.943000000000001,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5129999999999999,"5min":6.2370000000000001,"15min":2.8029999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.300999999999998,"5min":40.901000000000003,"15min":40.901000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":5.2530000000000001,"5min":5.7229999999999999,"15min":2.5539999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":17.434999999999999,"5min":33.162999999999997,"15min":39.137999999999998},"last":0.76600000000000001}]},{"osd":2,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.7759999999999998,"5min":6.0620000000000003,"15min":2.7469999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":18.202000000000002,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":5.8650000000000002,"5min":5.1820000000000004,"15min":2.4100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.893999999999998,"5min":25.616,"15min":58.899000000000001},"last":0.747}]},{"osd":3,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.9400000000000004,"5min":6.2759999999999998,"15min":2.6760000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":23.933,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":10.135,"5min":7.7439999999999998,"15min":3.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":34.786000000000001,"5min":111.221,"15min":111.221},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5339999999999998,"5min":5.5279999999999996,"15min":2.2120000000000002},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":20.184000000000001,"5min":51.871000000000002,"15min":51.871000000000002},"last":0.442},{"interface":"front","average":{"1min":3.6890000000000001,"5min":4.633,"15min":1.9590000000000001},"min":{"1min":0.47899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":23.858000000000001,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":4.5899999999999999,"5min":4.7770000000000001,"15min":1.9510000000000001},"min":{"1min":0.68999999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.905999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":5.673,"5min":5.524,"15min":2.2810000000000001},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":14.161,"5min":51.932000000000002,"15min":51.932000000000002},"last":0.495}]},{"osd":7,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9699999999999998,"5min":4.2699999999999996,"15min":1.8340000000000001},"min":{"1min":0.47899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":19.039000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":7.4729999999999999,"5min":5.6150000000000002,"15min":2.339},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":24.138000000000002,"5min":51.978000000000002,"15min":51.978000000000002},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.942,"5min":6.6719999999999997,"15min":2.6099999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":42.609000000000002,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":9.5069999999999997,"5min":5.5670000000000002,"15min":2.226},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":44.439,"5min":44.439,"15min":44.439},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.8230000000000004,"5min":6.2409999999999997,"15min":2.8530000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32100000000000001},"max":{"1min":31.355,"5min":32.787999999999997,"15min":124.85599999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":7.867,"5min":6,"15min":2.4079999999999999},"min":{"1min":0.48499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":31.385000000000002,"5min":41.060000000000002,"15min":41.060000000000002},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":12.061999999999999,"5min":7.4779999999999998,"15min":2.8919999999999999},"min":{"1min":0.46700000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":44.247,"5min":44.247,"15min":44.247},"last":0.69099999999999995},{"interface":"front","average":{"1min":8.5579999999999998,"5min":5.3840000000000003,"15min":2.23},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":36.612000000000002,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":19.018999999999998,"5min":7.3360000000000003,"15min":2.8769999999999998},"min":{"1min":0.49399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":79.974000000000004,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":12,"5min":10.103999999999999,"15min":3.8300000000000001},"min":{"1min":0.39900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":43.343000000000004,"5min":49.991999999999997,"15min":49.991999999999997},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.17,"5min":5.2290000000000001,"15min":2.117},"min":{"1min":0.69299999999999995,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":43.305,"5min":43.305,"15min":43.305},"last":0.40699999999999997},{"interface":"front","average":{"1min":9.3439999999999994,"5min":3.9980000000000002,"15min":1.7529999999999999},"min":{"1min":0.38600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":42.551000000000002,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.6980000000000004,"5min":7.0529999999999999,"15min":2.7629999999999999},"min":{"1min":0.53000000000000003,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":9.6489999999999991,"5min":5.4109999999999996,"15min":2.2120000000000002},"min":{"1min":0.60699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":46.505000000000003,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":8.0839999999999996,"5min":5.8360000000000003,"15min":2.4009999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":43.262,"5min":43.262,"15min":43.262},"last":0.48999999999999999},{"interface":"front","average":{"1min":9.3680000000000003,"5min":8.077,"15min":3.2120000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":44.018000000000001,"5min":80.302000000000007,"15min":80.302000000000007},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4621799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:41:12.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:41:12.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.865+0000 7f37ae1dd700 1 Processor -- start 2024-05-08T23:41:12.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.865+0000 7f37ae1dd700 1 -- start start 2024-05-08T23:41:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37ae1dd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f37a8162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37ae1dd700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a81582e0 con 0x7f37a80c6310 2024-05-08T23:41:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37ae1dd700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a8157e50 con 0x7f37a80c68e0 2024-05-08T23:41:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37ae1dd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8157310 con 0x7f37a8071e20 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37a77fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f37a80c6310 0x7f37a806baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40244/0 (socket says 172.21.15.33:40244) 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37a77fe700 1 -- 172.21.15.33:0/1880073107 learned_addr learned my addr 172.21.15.33:0/1880073107 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37a7fff700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f37a8162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.866+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138020317 0 0) 0x7f37a81582e0 con 0x7f37a80c6310 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37900035b0 con 0x7f37a80c6310 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314044649 0 0) 0x7f37900035b0 con 0x7f37a80c6310 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 msgr2=0x7f37a8162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f37a8162bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 >> v1:172.21.15.42:6789/0 conn(0x7f37a80c68e0 legacy=0x7f37a8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80c66f0 con 0x7f37a80c6310 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f3798001670 con 0x7f37a80c6310 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 >> v1:172.21.15.33:6789/0 conn(0x7f37a80c6310 legacy=0x7f37a806baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f3790003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f3790004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3790004d90 0x7f3790005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8157310 con 0x7f3790003a30 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379000bb50 con 0x7f3790004d90 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379000bd10 con 0x7f37a8071e20 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f3790003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a7fff700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3790004d90 0x7f3790005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a6ffd700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f3790004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 -- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 msgr2=0x7f3790004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8071e20 0x7f3790004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 -- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3790004d90 msgr2=0x7f3790005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3790004d90 0x7f3790005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.867+0000 7f37a77fe700 1 -- 172.21.15.33:0/1880073107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900035b0 con 0x7f3790003a30 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37a77fe700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f3790003e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f379800c270 tx=0x7f3790005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724f9fb449bf452d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798007c10 con 0x7f3790003a30 2024-05-08T23:41:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800cdb0 con 0x7f3790003a30 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37a67fc700 1 -- 172.21.15.33:0/1880073107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798005ba0 con 0x7f3790003a30 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 -- 172.21.15.33:0/1880073107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 msgr2=0x7f3790003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/1880073107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f3790003e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f379800c270 tx=0x7f3790005880 comp rx=0 tx=0).stop 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37a7fff700 1 -- 172.21.15.33:0/1880073107 reap_dead start 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 -- 172.21.15.33:0/1880073107 shutdown_connections 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 -- 172.21.15.33:0/1880073107 >> 172.21.15.33:0/1880073107 conn(0x7f37a806d2b0 msgr2=0x7f37a806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 -- 172.21.15.33:0/1880073107 shutdown_connections 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.868+0000 7f37ae1dd700 1 -- 172.21.15.33:0/1880073107 wait complete. 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.870+0000 7f37ae1dd700 1 Processor -- start 2024-05-08T23:41:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.870+0000 7f37ae1dd700 1 -- start start 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a80cbe40 0x7f37a814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8150310 0x7f37a8146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a814f9b0 con 0x7f3790003a30 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8152790 con 0x7f37a80cbe40 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37ae1dd700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a814dbd0 con 0x7f37a8150310 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44336/0 (socket says 172.21.15.33:44336) 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 -- 172.21.15.33:0/2547941394 learned_addr learned my addr 172.21.15.33:0/2547941394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a6ffd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a80cbe40 0x7f37a814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a7fff700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8150310 0x7f37a8146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 -- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8150310 msgr2=0x7f37a8146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8150310 0x7f37a8146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 -- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a80cbe40 msgr2=0x7f37a814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a80cbe40 0x7f37a814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 -- 172.21.15.33:0/2547941394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8149ec0 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f37a77fe700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3798001880 tx=0x7f37a80c4980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.871+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798001630 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.872+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37980070d0 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.872+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a81570b0 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.872+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798007440 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.872+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a814a420 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.873+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a81506f0 con 0x7f3790003a30 2024-05-08T23:41:12.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.873+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3798019030 con 0x7f3790003a30 2024-05-08T23:41:12.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.874+0000 7f378ffff700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 0x7f379406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:12.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.874+0000 7f37a6ffd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 0x7f379406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:12.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.874+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f379808a540 con 0x7f3790003a30 2024-05-08T23:41:12.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.874+0000 7f37a6ffd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 0x7f379406d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f37a8156080 tx=0x7f379c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:12.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:12.878+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3798055760 con 0x7f3790003a30 2024-05-08T23:41:13.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.095+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37a8157800 con 0x7f379406ac90 2024-05-08T23:41:13.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.103+0000 7f378ffff700 1 -- 172.21.15.33:0/2547941394 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290350 (secure 0 0 0) 0x7f379c001660 con 0x7f379406ac90 2024-05-08T23:41:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 msgr2=0x7f379406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 0x7f379406d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f37a8156080 tx=0x7f379c009040 comp rx=0 tx=0).stop 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 msgr2=0x7f37a8149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3798001880 tx=0x7f37a80c4980 comp rx=0 tx=0).stop 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 shutdown_connections 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f379406ac90 0x7f379406d150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f37a8150310 0x7f37a8146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3790003a30 0x7f37a8149470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 --2- 172.21.15.33:0/2547941394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37a80cbe40 0x7f37a814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 >> 172.21.15.33:0/2547941394 conn(0x7f37a806d2b0 msgr2=0x7f37a8072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 shutdown_connections 2024-05-08T23:41:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.107+0000 7f37ae1dd700 1 -- 172.21.15.33:0/2547941394 wait complete. 2024-05-08T23:41:13.110 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:41:13.118 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-05-08T23:41:11.494814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17883856,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320946,"num_write_kb":49019244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421683,"ondisk_log_size":421683,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4065176,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4028211,"kb_avail":750909544,"statfs":{"total":773094113280,"available":768931373056,"internally_reserved":0,"allocated":37773312,"data_stored":36471936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4124888632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003299"},"pg_stats":[{"pgid":"3.3f","version":"23'5694","reported_seq":5710,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.882557+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:40:50.882557+0000","last_peered":"2024-05-08T23:40:50.882557+0000","last_clean":"2024-05-08T23:40:50.882557+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:40:50.882557+0000","last_undegraded":"2024-05-08T23:40:50.882557+0000","last_fullsized":"2024-05-08T23:40:50.882557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8388","reported_seq":8404,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.810549+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:40:50.810549+0000","last_peered":"2024-05-08T23:40:50.810549+0000","last_clean":"2024-05-08T23:40:50.810549+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:40:50.810549+0000","last_undegraded":"2024-05-08T23:40:50.810549+0000","last_fullsized":"2024-05-08T23:40:50.810549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8388,"log_dups_size":0,"ondisk_log_size":8388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6507","reported_seq":6523,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.805708+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:40:50.805708+0000","last_peered":"2024-05-08T23:40:50.805708+0000","last_clean":"2024-05-08T23:40:50.805708+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:40:50.805708+0000","last_undegraded":"2024-05-08T23:40:50.805708+0000","last_fullsized":"2024-05-08T23:40:50.805708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6507,"log_dups_size":0,"ondisk_log_size":6507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5987","reported_seq":6003,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.725504+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:40:50.725504+0000","last_peered":"2024-05-08T23:40:50.725504+0000","last_clean":"2024-05-08T23:40:50.725504+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:40:50.725504+0000","last_undegraded":"2024-05-08T23:40:50.725504+0000","last_fullsized":"2024-05-08T23:40:50.725504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5987,"log_dups_size":0,"ondisk_log_size":5987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7603","reported_seq":7616,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885580+0000","last_change":"2024-05-08T23:32:59.855454+0000","last_active":"2024-05-08T23:40:50.885580+0000","last_peered":"2024-05-08T23:40:50.885580+0000","last_clean":"2024-05-08T23:40:50.885580+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:40:50.885580+0000","last_undegraded":"2024-05-08T23:40:50.885580+0000","last_fullsized":"2024-05-08T23:40:50.885580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7603,"log_dups_size":0,"ondisk_log_size":7603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:58.324852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5747","reported_seq":5762,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.540996+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:40:50.540996+0000","last_peered":"2024-05-08T23:40:50.540996+0000","last_clean":"2024-05-08T23:40:50.540996+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:40:50.540996+0000","last_undegraded":"2024-05-08T23:40:50.540996+0000","last_fullsized":"2024-05-08T23:40:50.540996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5747,"log_dups_size":0,"ondisk_log_size":5747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6492","reported_seq":6507,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.796565+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:40:50.796565+0000","last_peered":"2024-05-08T23:40:50.796565+0000","last_clean":"2024-05-08T23:40:50.796565+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:40:50.796565+0000","last_undegraded":"2024-05-08T23:40:50.796565+0000","last_fullsized":"2024-05-08T23:40:50.796565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6884","reported_seq":6899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849483+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:40:50.849483+0000","last_peered":"2024-05-08T23:40:50.849483+0000","last_clean":"2024-05-08T23:40:50.849483+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:40:50.849483+0000","last_undegraded":"2024-05-08T23:40:50.849483+0000","last_fullsized":"2024-05-08T23:40:50.849483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6884,"log_dups_size":0,"ondisk_log_size":6884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8081","reported_seq":8096,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.135086+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:40:50.135086+0000","last_peered":"2024-05-08T23:40:50.135086+0000","last_clean":"2024-05-08T23:40:50.135086+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:40:50.135086+0000","last_undegraded":"2024-05-08T23:40:50.135086+0000","last_fullsized":"2024-05-08T23:40:50.135086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7409","reported_seq":7424,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.785232+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:40:50.785232+0000","last_peered":"2024-05-08T23:40:50.785232+0000","last_clean":"2024-05-08T23:40:50.785232+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:40:50.785232+0000","last_undegraded":"2024-05-08T23:40:50.785232+0000","last_fullsized":"2024-05-08T23:40:50.785232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6316","reported_seq":6331,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.655933+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:40:50.655933+0000","last_peered":"2024-05-08T23:40:50.655933+0000","last_clean":"2024-05-08T23:40:50.655933+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:40:50.655933+0000","last_undegraded":"2024-05-08T23:40:50.655933+0000","last_fullsized":"2024-05-08T23:40:50.655933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5502","reported_seq":5517,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.801703+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:40:50.801703+0000","last_peered":"2024-05-08T23:40:50.801703+0000","last_clean":"2024-05-08T23:40:50.801703+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:40:50.801703+0000","last_undegraded":"2024-05-08T23:40:50.801703+0000","last_fullsized":"2024-05-08T23:40:50.801703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6029","reported_seq":6044,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.842402+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:40:50.842402+0000","last_peered":"2024-05-08T23:40:50.842402+0000","last_clean":"2024-05-08T23:40:50.842402+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:40:50.842402+0000","last_undegraded":"2024-05-08T23:40:50.842402+0000","last_fullsized":"2024-05-08T23:40:50.842402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6029,"log_dups_size":0,"ondisk_log_size":6029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6673","reported_seq":6688,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.804806+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:40:50.804806+0000","last_peered":"2024-05-08T23:40:50.804806+0000","last_clean":"2024-05-08T23:40:50.804806+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:40:50.804806+0000","last_undegraded":"2024-05-08T23:40:50.804806+0000","last_fullsized":"2024-05-08T23:40:50.804806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6514","reported_seq":6529,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836662+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:40:50.836662+0000","last_peered":"2024-05-08T23:40:50.836662+0000","last_clean":"2024-05-08T23:40:50.836662+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:40:50.836662+0000","last_undegraded":"2024-05-08T23:40:50.836662+0000","last_fullsized":"2024-05-08T23:40:50.836662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5772","reported_seq":5785,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.818932+0000","last_change":"2024-05-08T23:32:59.856316+0000","last_active":"2024-05-08T23:40:50.818932+0000","last_peered":"2024-05-08T23:40:50.818932+0000","last_clean":"2024-05-08T23:40:50.818932+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:40:50.818932+0000","last_undegraded":"2024-05-08T23:40:50.818932+0000","last_fullsized":"2024-05-08T23:40:50.818932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:19.763425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7973","reported_seq":7988,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.808102+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:40:50.808102+0000","last_peered":"2024-05-08T23:40:50.808102+0000","last_clean":"2024-05-08T23:40:50.808102+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:40:50.808102+0000","last_undegraded":"2024-05-08T23:40:50.808102+0000","last_fullsized":"2024-05-08T23:40:50.808102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7973,"log_dups_size":0,"ondisk_log_size":7973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5300","reported_seq":5315,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901529+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:40:50.901529+0000","last_peered":"2024-05-08T23:40:50.901529+0000","last_clean":"2024-05-08T23:40:50.901529+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:40:50.901529+0000","last_undegraded":"2024-05-08T23:40:50.901529+0000","last_fullsized":"2024-05-08T23:40:50.901529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6323","reported_seq":6338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.766273+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:40:50.766273+0000","last_peered":"2024-05-08T23:40:50.766273+0000","last_clean":"2024-05-08T23:40:50.766273+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:40:50.766273+0000","last_undegraded":"2024-05-08T23:40:50.766273+0000","last_fullsized":"2024-05-08T23:40:50.766273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'383","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.822638+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:40:50.822638+0000","last_peered":"2024-05-08T23:40:50.822638+0000","last_clean":"2024-05-08T23:40:50.822638+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:40:50.822638+0000","last_undegraded":"2024-05-08T23:40:50.822638+0000","last_fullsized":"2024-05-08T23:40:50.822638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6842","reported_seq":6856,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852287+0000","last_change":"2024-05-08T23:33:00.026361+0000","last_active":"2024-05-08T23:40:50.852287+0000","last_peered":"2024-05-08T23:40:50.852287+0000","last_clean":"2024-05-08T23:40:50.852287+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:40:50.852287+0000","last_undegraded":"2024-05-08T23:40:50.852287+0000","last_fullsized":"2024-05-08T23:40:50.852287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:26.768597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828674+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:40:50.828674+0000","last_peered":"2024-05-08T23:40:50.828674+0000","last_clean":"2024-05-08T23:40:50.828674+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:40:50.828674+0000","last_undegraded":"2024-05-08T23:40:50.828674+0000","last_fullsized":"2024-05-08T23:40:50.828674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7911","reported_seq":7935,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.906938+0000","last_change":"2024-05-08T23:32:59.859469+0000","last_active":"2024-05-08T23:40:50.906938+0000","last_peered":"2024-05-08T23:40:50.906938+0000","last_clean":"2024-05-08T23:40:50.906938+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:40:50.906938+0000","last_undegraded":"2024-05-08T23:40:50.906938+0000","last_fullsized":"2024-05-08T23:40:50.906938+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7911,"log_dups_size":0,"ondisk_log_size":7911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:35.269871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'10","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.837769+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:40:50.837769+0000","last_peered":"2024-05-08T23:40:50.837769+0000","last_clean":"2024-05-08T23:40:50.837769+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:40:50.837769+0000","last_undegraded":"2024-05-08T23:40:50.837769+0000","last_fullsized":"2024-05-08T23:40:50.837769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4519","reported_seq":4534,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.644012+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:40:50.644012+0000","last_peered":"2024-05-08T23:40:50.644012+0000","last_clean":"2024-05-08T23:40:50.644012+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:40:50.644012+0000","last_undegraded":"2024-05-08T23:40:50.644012+0000","last_fullsized":"2024-05-08T23:40:50.644012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.715531+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:40:50.715531+0000","last_peered":"2024-05-08T23:40:50.715531+0000","last_clean":"2024-05-08T23:40:50.715531+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:40:50.715531+0000","last_undegraded":"2024-05-08T23:40:50.715531+0000","last_fullsized":"2024-05-08T23:40:50.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7897","reported_seq":7912,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828353+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:40:50.828353+0000","last_peered":"2024-05-08T23:40:50.828353+0000","last_clean":"2024-05-08T23:40:50.828353+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:40:50.828353+0000","last_undegraded":"2024-05-08T23:40:50.828353+0000","last_fullsized":"2024-05-08T23:40:50.828353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1096","reported_seq":1112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.825119+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:40:50.825119+0000","last_peered":"2024-05-08T23:40:50.825119+0000","last_clean":"2024-05-08T23:40:50.825119+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:40:50.825119+0000","last_undegraded":"2024-05-08T23:40:50.825119+0000","last_fullsized":"2024-05-08T23:40:50.825119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5444","reported_seq":5459,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.736987+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:40:50.736987+0000","last_peered":"2024-05-08T23:40:50.736987+0000","last_clean":"2024-05-08T23:40:50.736987+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:40:50.736987+0000","last_undegraded":"2024-05-08T23:40:50.736987+0000","last_fullsized":"2024-05-08T23:40:50.736987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5444,"log_dups_size":0,"ondisk_log_size":5444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'25","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.878177+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:40:50.878177+0000","last_peered":"2024-05-08T23:40:50.878177+0000","last_clean":"2024-05-08T23:40:50.878177+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:40:50.878177+0000","last_undegraded":"2024-05-08T23:40:50.878177+0000","last_fullsized":"2024-05-08T23:40:50.878177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426823,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5057","reported_seq":5072,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.807049+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:40:50.807049+0000","last_peered":"2024-05-08T23:40:50.807049+0000","last_clean":"2024-05-08T23:40:50.807049+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:40:50.807049+0000","last_undegraded":"2024-05-08T23:40:50.807049+0000","last_fullsized":"2024-05-08T23:40:50.807049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836686+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:40:50.836686+0000","last_peered":"2024-05-08T23:40:50.836686+0000","last_clean":"2024-05-08T23:40:50.836686+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:40:50.836686+0000","last_undegraded":"2024-05-08T23:40:50.836686+0000","last_fullsized":"2024-05-08T23:40:50.836686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6593","reported_seq":6606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901627+0000","last_change":"2024-05-08T23:32:59.856036+0000","last_active":"2024-05-08T23:40:50.901627+0000","last_peered":"2024-05-08T23:40:50.901627+0000","last_clean":"2024-05-08T23:40:50.901627+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:40:50.901627+0000","last_undegraded":"2024-05-08T23:40:50.901627+0000","last_fullsized":"2024-05-08T23:40:50.901627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6593,"log_dups_size":0,"ondisk_log_size":6593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:07.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:37:06.070170+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:37:06.070170+0000","last_peered":"2024-05-08T23:37:06.070170+0000","last_clean":"2024-05-08T23:37:06.070170+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:37:06.070170+0000","last_undegraded":"2024-05-08T23:37:06.070170+0000","last_fullsized":"2024-05-08T23:37:06.070170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7580","reported_seq":7595,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.855374+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:40:50.855374+0000","last_peered":"2024-05-08T23:40:50.855374+0000","last_clean":"2024-05-08T23:40:50.855374+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:40:50.855374+0000","last_undegraded":"2024-05-08T23:40:50.855374+0000","last_fullsized":"2024-05-08T23:40:50.855374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7580,"log_dups_size":0,"ondisk_log_size":7580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.812887+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:40:50.812887+0000","last_peered":"2024-05-08T23:40:50.812887+0000","last_clean":"2024-05-08T23:40:50.812887+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:40:50.812887+0000","last_undegraded":"2024-05-08T23:40:50.812887+0000","last_fullsized":"2024-05-08T23:40:50.812887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6493","reported_seq":6508,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885950+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:40:50.885950+0000","last_peered":"2024-05-08T23:40:50.885950+0000","last_clean":"2024-05-08T23:40:50.885950+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:40:50.885950+0000","last_undegraded":"2024-05-08T23:40:50.885950+0000","last_fullsized":"2024-05-08T23:40:50.885950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6493,"log_dups_size":0,"ondisk_log_size":6493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.804197+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:40:50.804197+0000","last_peered":"2024-05-08T23:40:50.804197+0000","last_clean":"2024-05-08T23:40:50.804197+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:40:50.804197+0000","last_undegraded":"2024-05-08T23:40:50.804197+0000","last_fullsized":"2024-05-08T23:40:50.804197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5812","reported_seq":5835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.807383+0000","last_change":"2024-05-08T23:32:59.859486+0000","last_active":"2024-05-08T23:40:50.807383+0000","last_peered":"2024-05-08T23:40:50.807383+0000","last_clean":"2024-05-08T23:40:50.807383+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:40:50.807383+0000","last_undegraded":"2024-05-08T23:40:50.807383+0000","last_fullsized":"2024-05-08T23:40:50.807383+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:23.076314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'17","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869954+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:40:50.869954+0000","last_peered":"2024-05-08T23:40:50.869954+0000","last_clean":"2024-05-08T23:40:50.869954+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:40:50.869954+0000","last_undegraded":"2024-05-08T23:40:50.869954+0000","last_fullsized":"2024-05-08T23:40:50.869954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8441","reported_seq":8456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885980+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:40:50.885980+0000","last_peered":"2024-05-08T23:40:50.885980+0000","last_clean":"2024-05-08T23:40:50.885980+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:40:50.885980+0000","last_undegraded":"2024-05-08T23:40:50.885980+0000","last_fullsized":"2024-05-08T23:40:50.885980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8441,"log_dups_size":0,"ondisk_log_size":8441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'222","reported_seq":238,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.718826+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:40:50.718826+0000","last_peered":"2024-05-08T23:40:50.718826+0000","last_clean":"2024-05-08T23:40:50.718826+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:40:50.718826+0000","last_undegraded":"2024-05-08T23:40:50.718826+0000","last_fullsized":"2024-05-08T23:40:50.718826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6150","reported_seq":6165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.633129+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:40:50.633129+0000","last_peered":"2024-05-08T23:40:50.633129+0000","last_clean":"2024-05-08T23:40:50.633129+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:40:50.633129+0000","last_undegraded":"2024-05-08T23:40:50.633129+0000","last_fullsized":"2024-05-08T23:40:50.633129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6150,"log_dups_size":0,"ondisk_log_size":6150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854210+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:40:50.854210+0000","last_peered":"2024-05-08T23:40:50.854210+0000","last_clean":"2024-05-08T23:40:50.854210+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:40:50.854210+0000","last_undegraded":"2024-05-08T23:40:50.854210+0000","last_fullsized":"2024-05-08T23:40:50.854210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5975","reported_seq":5990,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.722519+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:40:50.722519+0000","last_peered":"2024-05-08T23:40:50.722519+0000","last_clean":"2024-05-08T23:40:50.722519+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:40:50.722519+0000","last_undegraded":"2024-05-08T23:40:50.722519+0000","last_fullsized":"2024-05-08T23:40:50.722519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5975,"log_dups_size":0,"ondisk_log_size":5975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.883912+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:40:50.883912+0000","last_peered":"2024-05-08T23:40:50.883912+0000","last_clean":"2024-05-08T23:40:50.883912+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:40:50.883912+0000","last_undegraded":"2024-05-08T23:40:50.883912+0000","last_fullsized":"2024-05-08T23:40:50.883912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7310","reported_seq":7325,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.859323+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:40:50.859323+0000","last_peered":"2024-05-08T23:40:50.859323+0000","last_clean":"2024-05-08T23:40:50.859323+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:40:50.859323+0000","last_undegraded":"2024-05-08T23:40:50.859323+0000","last_fullsized":"2024-05-08T23:40:50.859323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7310,"log_dups_size":0,"ondisk_log_size":7310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.831450+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:40:50.831450+0000","last_peered":"2024-05-08T23:40:50.831450+0000","last_clean":"2024-05-08T23:40:50.831450+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:40:50.831450+0000","last_undegraded":"2024-05-08T23:40:50.831450+0000","last_fullsized":"2024-05-08T23:40:50.831450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7011","reported_seq":7026,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.714630+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:40:50.714630+0000","last_peered":"2024-05-08T23:40:50.714630+0000","last_clean":"2024-05-08T23:40:50.714630+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:40:50.714630+0000","last_undegraded":"2024-05-08T23:40:50.714630+0000","last_fullsized":"2024-05-08T23:40:50.714630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'15","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828797+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:40:50.828797+0000","last_peered":"2024-05-08T23:40:50.828797+0000","last_clean":"2024-05-08T23:40:50.828797+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:40:50.828797+0000","last_undegraded":"2024-05-08T23:40:50.828797+0000","last_fullsized":"2024-05-08T23:40:50.828797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6605","reported_seq":6620,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.792042+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:40:50.792042+0000","last_peered":"2024-05-08T23:40:50.792042+0000","last_clean":"2024-05-08T23:40:50.792042+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:40:50.792042+0000","last_undegraded":"2024-05-08T23:40:50.792042+0000","last_fullsized":"2024-05-08T23:40:50.792042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836668+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:40:50.836668+0000","last_peered":"2024-05-08T23:40:50.836668+0000","last_clean":"2024-05-08T23:40:50.836668+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:40:50.836668+0000","last_undegraded":"2024-05-08T23:40:50.836668+0000","last_fullsized":"2024-05-08T23:40:50.836668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6071","reported_seq":6094,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870424+0000","last_change":"2024-05-08T23:33:00.026523+0000","last_active":"2024-05-08T23:40:50.870424+0000","last_peered":"2024-05-08T23:40:50.870424+0000","last_clean":"2024-05-08T23:40:50.870424+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:40:50.870424+0000","last_undegraded":"2024-05-08T23:40:50.870424+0000","last_fullsized":"2024-05-08T23:40:50.870424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6071,"log_dups_size":0,"ondisk_log_size":6071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:23.920007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826077+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:40:50.826077+0000","last_peered":"2024-05-08T23:40:50.826077+0000","last_clean":"2024-05-08T23:40:50.826077+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:40:50.826077+0000","last_undegraded":"2024-05-08T23:40:50.826077+0000","last_fullsized":"2024-05-08T23:40:50.826077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5809","reported_seq":5825,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854937+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:40:50.854937+0000","last_peered":"2024-05-08T23:40:50.854937+0000","last_clean":"2024-05-08T23:40:50.854937+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:40:50.854937+0000","last_undegraded":"2024-05-08T23:40:50.854937+0000","last_fullsized":"2024-05-08T23:40:50.854937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:27.185535+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:38:27.185535+0000","last_peered":"2024-05-08T23:38:27.185535+0000","last_clean":"2024-05-08T23:38:27.185535+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:38:27.185535+0000","last_undegraded":"2024-05-08T23:38:27.185535+0000","last_fullsized":"2024-05-08T23:38:27.185535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6329","reported_seq":6345,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.684184+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:40:50.684184+0000","last_peered":"2024-05-08T23:40:50.684184+0000","last_clean":"2024-05-08T23:40:50.684184+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:40:50.684184+0000","last_undegraded":"2024-05-08T23:40:50.684184+0000","last_fullsized":"2024-05-08T23:40:50.684184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6273","reported_seq":6289,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.844273+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:40:50.844273+0000","last_peered":"2024-05-08T23:40:50.844273+0000","last_clean":"2024-05-08T23:40:50.844273+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:40:50.844273+0000","last_undegraded":"2024-05-08T23:40:50.844273+0000","last_fullsized":"2024-05-08T23:40:50.844273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6273,"log_dups_size":0,"ondisk_log_size":6273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'27","reported_seq":68,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828876+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:40:50.828876+0000","last_peered":"2024-05-08T23:40:50.828876+0000","last_clean":"2024-05-08T23:40:50.828876+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:40:50.828876+0000","last_undegraded":"2024-05-08T23:40:50.828876+0000","last_fullsized":"2024-05-08T23:40:50.828876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6847","reported_seq":6863,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.265524+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:40:50.265524+0000","last_peered":"2024-05-08T23:40:50.265524+0000","last_clean":"2024-05-08T23:40:50.265524+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:40:50.265524+0000","last_undegraded":"2024-05-08T23:40:50.265524+0000","last_fullsized":"2024-05-08T23:40:50.265524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.805976+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:40:50.805976+0000","last_peered":"2024-05-08T23:40:50.805976+0000","last_clean":"2024-05-08T23:40:50.805976+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:40:50.805976+0000","last_undegraded":"2024-05-08T23:40:50.805976+0000","last_fullsized":"2024-05-08T23:40:50.805976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:16.309946+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:38:16.309946+0000","last_peered":"2024-05-08T23:38:16.309946+0000","last_clean":"2024-05-08T23:38:16.309946+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:38:16.309946+0000","last_undegraded":"2024-05-08T23:38:16.309946+0000","last_fullsized":"2024-05-08T23:38:16.309946+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6391","reported_seq":6407,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.792470+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:40:50.792470+0000","last_peered":"2024-05-08T23:40:50.792470+0000","last_clean":"2024-05-08T23:40:50.792470+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:40:50.792470+0000","last_undegraded":"2024-05-08T23:40:50.792470+0000","last_fullsized":"2024-05-08T23:40:50.792470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6391,"log_dups_size":0,"ondisk_log_size":6391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849138+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:40:50.849138+0000","last_peered":"2024-05-08T23:40:50.849138+0000","last_clean":"2024-05-08T23:40:50.849138+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:40:50.849138+0000","last_undegraded":"2024-05-08T23:40:50.849138+0000","last_fullsized":"2024-05-08T23:40:50.849138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:38.734890+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:40:38.734890+0000","last_peered":"2024-05-08T23:40:38.734890+0000","last_clean":"2024-05-08T23:40:38.734890+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:40:38.734890+0000","last_undegraded":"2024-05-08T23:40:38.734890+0000","last_fullsized":"2024-05-08T23:40:38.734890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7006","reported_seq":7022,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.821144+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:40:50.821144+0000","last_peered":"2024-05-08T23:40:50.821144+0000","last_clean":"2024-05-08T23:40:50.821144+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:40:50.821144+0000","last_undegraded":"2024-05-08T23:40:50.821144+0000","last_fullsized":"2024-05-08T23:40:50.821144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7006,"log_dups_size":0,"ondisk_log_size":7006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'86","reported_seq":103,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826703+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:40:50.826703+0000","last_peered":"2024-05-08T23:40:50.826703+0000","last_clean":"2024-05-08T23:40:50.826703+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:40:50.826703+0000","last_undegraded":"2024-05-08T23:40:50.826703+0000","last_fullsized":"2024-05-08T23:40:50.826703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6426","reported_seq":6442,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.839221+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:40:50.839221+0000","last_peered":"2024-05-08T23:40:50.839221+0000","last_clean":"2024-05-08T23:40:50.839221+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:40:50.839221+0000","last_undegraded":"2024-05-08T23:40:50.839221+0000","last_fullsized":"2024-05-08T23:40:50.839221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6426,"log_dups_size":0,"ondisk_log_size":6426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.423772+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:40:51.423772+0000","last_peered":"2024-05-08T23:40:51.423772+0000","last_clean":"2024-05-08T23:40:51.423772+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:40:51.423772+0000","last_undegraded":"2024-05-08T23:40:51.423772+0000","last_fullsized":"2024-05-08T23:40:51.423772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5451","reported_seq":5467,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.795395+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:40:50.795395+0000","last_peered":"2024-05-08T23:40:50.795395+0000","last_clean":"2024-05-08T23:40:50.795395+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:40:50.795395+0000","last_undegraded":"2024-05-08T23:40:50.795395+0000","last_fullsized":"2024-05-08T23:40:50.795395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.851008+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:40:50.851008+0000","last_peered":"2024-05-08T23:40:50.851008+0000","last_clean":"2024-05-08T23:40:50.851008+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:40:50.851008+0000","last_undegraded":"2024-05-08T23:40:50.851008+0000","last_fullsized":"2024-05-08T23:40:50.851008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5843","reported_seq":5859,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870876+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:40:50.870876+0000","last_peered":"2024-05-08T23:40:50.870876+0000","last_clean":"2024-05-08T23:40:50.870876+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:40:50.870876+0000","last_undegraded":"2024-05-08T23:40:50.870876+0000","last_fullsized":"2024-05-08T23:40:50.870876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5843,"log_dups_size":0,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.835933+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:40:50.835933+0000","last_peered":"2024-05-08T23:40:50.835933+0000","last_clean":"2024-05-08T23:40:50.835933+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:40:50.835933+0000","last_undegraded":"2024-05-08T23:40:50.835933+0000","last_fullsized":"2024-05-08T23:40:50.835933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6479","reported_seq":6495,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.624049+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:40:50.624049+0000","last_peered":"2024-05-08T23:40:50.624049+0000","last_clean":"2024-05-08T23:40:50.624049+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:40:50.624049+0000","last_undegraded":"2024-05-08T23:40:50.624049+0000","last_fullsized":"2024-05-08T23:40:50.624049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6479,"log_dups_size":0,"ondisk_log_size":6479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.875794+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:40:50.875794+0000","last_peered":"2024-05-08T23:40:50.875794+0000","last_clean":"2024-05-08T23:40:50.875794+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:40:50.875794+0000","last_undegraded":"2024-05-08T23:40:50.875794+0000","last_fullsized":"2024-05-08T23:40:50.875794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5333","reported_seq":5348,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.796682+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:40:50.796682+0000","last_peered":"2024-05-08T23:40:50.796682+0000","last_clean":"2024-05-08T23:40:50.796682+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:40:50.796682+0000","last_undegraded":"2024-05-08T23:40:50.796682+0000","last_fullsized":"2024-05-08T23:40:50.796682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5333,"log_dups_size":0,"ondisk_log_size":5333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'685","reported_seq":701,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852470+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:40:50.852470+0000","last_peered":"2024-05-08T23:40:50.852470+0000","last_clean":"2024-05-08T23:40:50.852470+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:40:50.852470+0000","last_undegraded":"2024-05-08T23:40:50.852470+0000","last_fullsized":"2024-05-08T23:40:50.852470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5933","reported_seq":5949,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.871976+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:40:50.871976+0000","last_peered":"2024-05-08T23:40:50.871976+0000","last_clean":"2024-05-08T23:40:50.871976+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:40:50.871976+0000","last_undegraded":"2024-05-08T23:40:50.871976+0000","last_fullsized":"2024-05-08T23:40:50.871976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5933,"log_dups_size":0,"ondisk_log_size":5933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.896708+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:40:50.896708+0000","last_peered":"2024-05-08T23:40:50.896708+0000","last_clean":"2024-05-08T23:40:50.896708+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:40:50.896708+0000","last_undegraded":"2024-05-08T23:40:50.896708+0000","last_fullsized":"2024-05-08T23:40:50.896708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6537","reported_seq":6552,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.899748+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:40:50.899748+0000","last_peered":"2024-05-08T23:40:50.899748+0000","last_clean":"2024-05-08T23:40:50.899748+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:40:50.899748+0000","last_undegraded":"2024-05-08T23:40:50.899748+0000","last_fullsized":"2024-05-08T23:40:50.899748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6537,"log_dups_size":0,"ondisk_log_size":6537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:39:01.910099+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:39:01.910099+0000","last_peered":"2024-05-08T23:39:01.910099+0000","last_clean":"2024-05-08T23:39:01.910099+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:39:01.910099+0000","last_undegraded":"2024-05-08T23:39:01.910099+0000","last_fullsized":"2024-05-08T23:39:01.910099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6818","reported_seq":6833,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.897583+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:40:50.897583+0000","last_peered":"2024-05-08T23:40:50.897583+0000","last_clean":"2024-05-08T23:40:50.897583+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:40:50.897583+0000","last_undegraded":"2024-05-08T23:40:50.897583+0000","last_fullsized":"2024-05-08T23:40:50.897583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6818,"log_dups_size":0,"ondisk_log_size":6818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828990+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:40:50.828990+0000","last_peered":"2024-05-08T23:40:50.828990+0000","last_clean":"2024-05-08T23:40:50.828990+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:40:50.828990+0000","last_undegraded":"2024-05-08T23:40:50.828990+0000","last_fullsized":"2024-05-08T23:40:50.828990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7691","reported_seq":7706,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.671715+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:40:50.671715+0000","last_peered":"2024-05-08T23:40:50.671715+0000","last_clean":"2024-05-08T23:40:50.671715+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:40:50.671715+0000","last_undegraded":"2024-05-08T23:40:50.671715+0000","last_fullsized":"2024-05-08T23:40:50.671715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7797","reported_seq":7813,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.827905+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:40:50.827905+0000","last_peered":"2024-05-08T23:40:50.827905+0000","last_clean":"2024-05-08T23:40:50.827905+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:40:50.827905+0000","last_undegraded":"2024-05-08T23:40:50.827905+0000","last_fullsized":"2024-05-08T23:40:50.827905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7797,"log_dups_size":0,"ondisk_log_size":7797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6562","reported_seq":6578,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.602602+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:40:50.602602+0000","last_peered":"2024-05-08T23:40:50.602602+0000","last_clean":"2024-05-08T23:40:50.602602+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:40:50.602602+0000","last_undegraded":"2024-05-08T23:40:50.602602+0000","last_fullsized":"2024-05-08T23:40:50.602602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8077","reported_seq":8093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.884270+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:40:50.884270+0000","last_peered":"2024-05-08T23:40:50.884270+0000","last_clean":"2024-05-08T23:40:50.884270+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:40:50.884270+0000","last_undegraded":"2024-05-08T23:40:50.884270+0000","last_fullsized":"2024-05-08T23:40:50.884270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5650","reported_seq":5666,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.745990+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:40:50.745990+0000","last_peered":"2024-05-08T23:40:50.745990+0000","last_clean":"2024-05-08T23:40:50.745990+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:40:50.745990+0000","last_undegraded":"2024-05-08T23:40:50.745990+0000","last_fullsized":"2024-05-08T23:40:50.745990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5650,"log_dups_size":0,"ondisk_log_size":5650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6013","reported_seq":6029,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.789472+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:40:50.789472+0000","last_peered":"2024-05-08T23:40:50.789472+0000","last_clean":"2024-05-08T23:40:50.789472+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:40:50.789472+0000","last_undegraded":"2024-05-08T23:40:50.789472+0000","last_fullsized":"2024-05-08T23:40:50.789472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7047","reported_seq":7063,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.900550+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:40:50.900550+0000","last_peered":"2024-05-08T23:40:50.900550+0000","last_clean":"2024-05-08T23:40:50.900550+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:40:50.900550+0000","last_undegraded":"2024-05-08T23:40:50.900550+0000","last_fullsized":"2024-05-08T23:40:50.900550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5651","reported_seq":5667,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.880110+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:40:50.880110+0000","last_peered":"2024-05-08T23:40:50.880110+0000","last_clean":"2024-05-08T23:40:50.880110+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:40:50.880110+0000","last_undegraded":"2024-05-08T23:40:50.880110+0000","last_fullsized":"2024-05-08T23:40:50.880110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5651,"log_dups_size":0,"ondisk_log_size":5651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7696","reported_seq":7712,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.880040+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:40:50.880040+0000","last_peered":"2024-05-08T23:40:50.880040+0000","last_clean":"2024-05-08T23:40:50.880040+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:40:50.880040+0000","last_undegraded":"2024-05-08T23:40:50.880040+0000","last_fullsized":"2024-05-08T23:40:50.880040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7696,"log_dups_size":0,"ondisk_log_size":7696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5551","reported_seq":5567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869705+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:40:50.869705+0000","last_peered":"2024-05-08T23:40:50.869705+0000","last_clean":"2024-05-08T23:40:50.869705+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:40:50.869705+0000","last_undegraded":"2024-05-08T23:40:50.869705+0000","last_fullsized":"2024-05-08T23:40:50.869705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5551,"log_dups_size":0,"ondisk_log_size":5551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6385","reported_seq":6400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836597+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:40:50.836597+0000","last_peered":"2024-05-08T23:40:50.836597+0000","last_clean":"2024-05-08T23:40:50.836597+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:40:50.836597+0000","last_undegraded":"2024-05-08T23:40:50.836597+0000","last_fullsized":"2024-05-08T23:40:50.836597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6385,"log_dups_size":0,"ondisk_log_size":6385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7231","reported_seq":7246,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.609971+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:40:50.609971+0000","last_peered":"2024-05-08T23:40:50.609971+0000","last_clean":"2024-05-08T23:40:50.609971+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:40:50.609971+0000","last_undegraded":"2024-05-08T23:40:50.609971+0000","last_fullsized":"2024-05-08T23:40:50.609971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7231,"log_dups_size":0,"ondisk_log_size":7231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6799","reported_seq":6814,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.884006+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:40:50.884006+0000","last_peered":"2024-05-08T23:40:50.884006+0000","last_clean":"2024-05-08T23:40:50.884006+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:40:50.884006+0000","last_undegraded":"2024-05-08T23:40:50.884006+0000","last_fullsized":"2024-05-08T23:40:50.884006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6386","reported_seq":6401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.814908+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:40:50.814908+0000","last_peered":"2024-05-08T23:40:50.814908+0000","last_clean":"2024-05-08T23:40:50.814908+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:40:50.814908+0000","last_undegraded":"2024-05-08T23:40:50.814908+0000","last_fullsized":"2024-05-08T23:40:50.814908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":418916,"ondisk_log_size":418916,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293488,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2678,"num_write_kb":19121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34586976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2650,"ondisk_log_size":2650,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633788,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":620670,"kb_avail":93738052,"statfs":{"total":96636764160,"available":95987765248,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":635566539},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":8.4469999999999992,"15min":3.2210000000000001},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":45.125999999999998,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":9.0259999999999998,"5min":7.827,"15min":3.0419999999999998},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":37.293999999999997,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.194,"5min":4.4950000000000001,"15min":1.9470000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":13.531000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":9.1940000000000008,"5min":7.9210000000000003,"15min":3.1160000000000001},"min":{"1min":0.40000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":24.387,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.5529999999999999,"5min":4.2610000000000001,"15min":1.8879999999999999},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":15.938000000000001,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":6.0819999999999999,"5min":5.0229999999999997,"15min":2.1219999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":19.565000000000001,"5min":26.084,"15min":26.084},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":6.0049999999999999,"5min":7.1989999999999998,"15min":2.8839999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":19.614000000000001,"5min":52.869,"15min":52.869},"last":0.52900000000000003},{"interface":"front","average":{"1min":10.468,"5min":7.2720000000000002,"15min":2.855},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35499999999999998},"max":{"1min":49.421999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.67500000000000004}]},{"osd":4,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":5.0599999999999996,"15min":2.0430000000000001},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.9139999999999997,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":5.3949999999999996,"5min":3.8919999999999999,"15min":1.6950000000000001},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":22.901,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":7.1719999999999997,"5min":5.1349999999999998,"15min":2.0910000000000002},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":18.988,"5min":63.732999999999997,"15min":63.732999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":9.2859999999999996,"5min":7.7160000000000002,"15min":2.9860000000000002},"min":{"1min":0.69199999999999995,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":18.504999999999999,"5min":62.558,"15min":62.558},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":10.367000000000001,"5min":7.5259999999999998,"15min":2.9529999999999998},"min":{"1min":0.71799999999999997,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":45.826999999999998,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":5.8150000000000004,"5min":4.6600000000000001,"15min":1.982},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40600000000000003},"max":{"1min":38.801000000000002,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.60499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4709827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.7320000000000002,"5min":6.8529999999999998,"15min":2.7749999999999999},"min":{"1min":0.52500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":19.369,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":6.4900000000000002,"5min":7.8170000000000002,"15min":3.1259999999999999},"min":{"1min":0.47199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":18.379000000000001,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":6.8890000000000002,"5min":6.3970000000000002,"15min":2.8570000000000002},"min":{"1min":0.876,"5min":0.313,"15min":0.313},"max":{"1min":24.693000000000001,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":5.6459999999999999,"5min":5.7210000000000001,"15min":2.6230000000000002},"min":{"1min":0.57899999999999996,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":24.187999999999999,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":4.306,"5min":4.7060000000000004,"15min":2.1030000000000002},"min":{"1min":0.58499999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":15.589,"5min":41.445,"15min":41.445},"last":0.69799999999999995},{"interface":"front","average":{"1min":4.3399999999999999,"5min":4.6550000000000002,"15min":2.0630000000000002},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":15.624000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.77400000000000002}]},{"osd":3,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.4909999999999997,"5min":5.2729999999999997,"15min":2.266},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":15.467000000000001,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":5.8230000000000004,"5min":7.4550000000000001,"15min":3.1749999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":28.600000000000001,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":3.141,"15min":1.446},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.891,"5min":20.913,"15min":20.913},"last":0.5},{"interface":"front","average":{"1min":2.1909999999999998,"5min":3.875,"15min":1.8129999999999999},"min":{"1min":0.52700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.9569999999999999,"5min":35.975999999999999,"15min":35.975999999999999},"last":0.752}]},{"osd":5,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":7.3280000000000003,"5min":6.5369999999999999,"15min":2.657},"min":{"1min":0.64500000000000002,"5min":0.39200000000000002,"15min":0.33100000000000002},"max":{"1min":36.756,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.624},{"interface":"front","average":{"1min":6.0759999999999996,"5min":4.633,"15min":2.0230000000000001},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":23.105,"5min":23.105,"15min":23.105},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":4.2359999999999998,"15min":2.0259999999999998},"min":{"1min":0.44800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":6.9900000000000002,"5min":24.16,"15min":33.463000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":5.7530000000000001,"5min":5.6429999999999998,"15min":2.3460000000000001},"min":{"1min":0.83299999999999996,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.056999999999999,"5min":45.805,"15min":45.805},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":7.673,"5min":6.6539999999999999,"15min":2.5219999999999998},"min":{"1min":0.437,"5min":0.29099999999999998,"15min":0.23400000000000001},"max":{"1min":30.721,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":4.6159999999999997,"5min":5.1420000000000003,"15min":2.1499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":13.93,"5min":48.844999999999999,"15min":48.844999999999999},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.6609999999999996,"5min":4.5510000000000002,"15min":1.897},"min":{"1min":0.32400000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":22.721,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.5819999999999999,"15min":2.3359999999999999},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":25.683,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":7.8360000000000003,"15min":2.9780000000000002},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":21.975000000000001,"5min":52.712000000000003,"15min":52.712000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":4.9900000000000002,"5min":9.1709999999999994,"15min":3.4780000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.32100000000000001},"max":{"1min":25.724,"5min":61.832000000000001,"15min":61.832000000000001},"last":0.70399999999999996}]},{"osd":4,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":6.2939999999999996,"5min":6.4749999999999996,"15min":2.6850000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":23.529,"5min":45.267000000000003,"15min":45.267000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":7.2350000000000003,"5min":6.3070000000000004,"15min":2.5979999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":31.245999999999999,"5min":43.353999999999999,"15min":43.353999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":5.3419999999999996,"5min":6.3719999999999999,"15min":2.6850000000000001},"min":{"1min":0.66300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":16.907,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":6.984,"5min":8.1400000000000006,"15min":3.2719999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":36.280999999999999,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.4470000000000001,"5min":7.5030000000000001,"15min":3.0939999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":11.292999999999999,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":6.8490000000000002,"5min":7.016,"15min":2.8900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":25.600999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":10.384,"5min":8.8420000000000005,"15min":3.4609999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":72.088999999999999,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":10.420999999999999,"5min":9.4009999999999998,"15min":3.7349999999999999},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":72.150000000000006,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.628}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.324,"5min":6.6020000000000003,"15min":2.8780000000000001},"min":{"1min":0.28599999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":47.241999999999997,"5min":47.241999999999997,"15min":101.899},"last":0.47199999999999998},{"interface":"front","average":{"1min":7.3029999999999999,"5min":5.3879999999999999,"15min":2.5339999999999998},"min":{"1min":0.65000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":22.170999999999999,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.44}]},{"osd":2,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":14.221,"5min":8.1920000000000002,"15min":3.5609999999999999},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":134.16300000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":10.577999999999999,"5min":7.7560000000000002,"15min":3.3380000000000001},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":47.277000000000001,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.8740000000000006,"5min":7.0720000000000001,"15min":3.1829999999999998},"min":{"1min":0.59399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":59.796999999999997,"5min":59.796999999999997,"15min":101.929},"last":0.52100000000000002},{"interface":"front","average":{"1min":4.3179999999999996,"5min":4.8780000000000001,"15min":2.2130000000000001},"min":{"1min":0.47499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.606,"5min":95.200999999999993,"15min":95.200999999999993},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.207000000000001,"5min":6.8620000000000001,"15min":3.206},"min":{"1min":0.79500000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":47.194000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.70499999999999996},{"interface":"front","average":{"1min":4.2290000000000001,"5min":5.282,"15min":2.3500000000000001},"min":{"1min":0.54300000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":12.694000000000001,"5min":39.813000000000002,"15min":44.545999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.326000000000001,"5min":6.3300000000000001,"15min":2.8500000000000001},"min":{"1min":1.111,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":38.494,"5min":38.494,"15min":78.780000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":5.0940000000000003,"5min":6.5419999999999998,"15min":2.7690000000000001},"min":{"1min":0.79800000000000004,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":12.657999999999999,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.0920000000000005,"5min":7.1189999999999998,"15min":3.2959999999999998},"min":{"1min":0.85299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":45.744999999999997,"5min":74.013000000000005,"15min":101.869},"last":0.66000000000000003},{"interface":"front","average":{"1min":10.259,"5min":6.2560000000000002,"15min":2.903},"min":{"1min":0.46100000000000002,"5min":0.317,"15min":0.317},"max":{"1min":38.545000000000002,"5min":71.323999999999998,"15min":78.647999999999996},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.394,"5min":6,"15min":2.8479999999999999},"min":{"1min":0.67200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":37.725000000000001,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.80600000000000005},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.9059999999999997,"15min":2.2759999999999998},"min":{"1min":0.61199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":13.746,"5min":27.056000000000001,"15min":44.713999999999999},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":5.6520000000000001,"5min":5.6859999999999999,"15min":2.1960000000000002},"min":{"1min":0.35499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":44.238999999999997,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":6.6989999999999998,"5min":5.024,"15min":2.0939999999999999},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.27700000000000002},"max":{"1min":44.353000000000002,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":10.422000000000001,"5min":6.5999999999999996,"15min":2.5630000000000002},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.255},"max":{"1min":79.540999999999997,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":11.664,"5min":6.3029999999999999,"15min":2.476},"min":{"1min":0.52100000000000002,"5min":0.374,"15min":0.318},"max":{"1min":79.584000000000003,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":8.9100000000000001,"5min":9.1769999999999996,"15min":3.3730000000000002},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":26.959,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":3.363,"5min":5.0359999999999996,"15min":2.0699999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":18.030999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.4589999999999996,"5min":5.0419999999999998,"15min":2.169},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":41.488999999999997,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":4.4950000000000001,"5min":4.5179999999999998,"15min":1.9850000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":17.946999999999999,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.4720000000000004,"5min":6.577,"15min":2.7519999999999998},"min":{"1min":0.88700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":24.843,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":5.4720000000000004,"5min":5.3280000000000003,"15min":2.2970000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":14.734999999999999,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.2080000000000002,"5min":5.0599999999999996,"15min":2.1970000000000001},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":41.396999999999998,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":7.5640000000000001,"5min":6.2050000000000001,"15min":2.5430000000000001},"min":{"1min":0.93999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":24.686,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.2050000000000001,"5min":5.2779999999999996,"15min":2.3100000000000001},"min":{"1min":0.70299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":24.623999999999999,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":4.6799999999999997,"5min":7.0670000000000002,"15min":2.8570000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":17.905000000000001,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":4.125,"5min":6.7089999999999996,"15min":2.5619999999999998},"min":{"1min":0.35299999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":18.997,"5min":67.305999999999997,"15min":67.305999999999997},"last":0.30099999999999999},{"interface":"front","average":{"1min":6.2910000000000004,"5min":9.1940000000000008,"15min":3.3969999999999998},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.149999999999999,"5min":106.934,"15min":106.934},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":9.6839999999999993,"5min":7.0439999999999996,"15min":2.6669999999999998},"min":{"1min":0.40699999999999997,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":76.628,"5min":76.628,"15min":76.628},"last":0.35299999999999998},{"interface":"front","average":{"1min":4.9690000000000003,"5min":6.4690000000000003,"15min":2.5129999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":19.045999999999999,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":8.5600000000000005,"5min":7.0899999999999999,"15min":3.008},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":57.378,"5min":57.378,"15min":57.378},"last":0.36699999999999999},{"interface":"front","average":{"1min":4.3460000000000001,"5min":5.8959999999999999,"15min":2.363},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":19.800000000000001,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":11.138999999999999,"5min":8.3179999999999996,"15min":3.2629999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":57.313000000000002,"5min":57.313000000000002,"15min":57.313000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":11.608000000000001,"5min":7.4400000000000004,"15min":2.9350000000000001},"min":{"1min":1.5469999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":53.343000000000004,"5min":68.238,"15min":68.238},"last":0.70499999999999996}]},{"osd":5,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.965,"5min":6.7839999999999998,"15min":2.722},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":53.381,"5min":53.381,"15min":53.381},"last":0.79400000000000004},{"interface":"front","average":{"1min":6.71,"5min":7.867,"15min":3.1309999999999998},"min":{"1min":0.47099999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":19.084,"5min":36.213999999999999,"15min":36.213999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.84,"5min":7.8650000000000002,"15min":3.0960000000000001},"min":{"1min":1.2270000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":57.411000000000001,"5min":57.411000000000001,"15min":57.411000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":6.2850000000000001,"5min":5.6790000000000003,"15min":2.3849999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":19.113,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":9.2200000000000006,"15min":3.5830000000000002},"min":{"1min":0.59399999999999997,"5min":0.375,"15min":0.375},"max":{"1min":63.896999999999998,"5min":63.896999999999998,"15min":63.896999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":9.6880000000000006,"5min":7.1580000000000004,"15min":2.9129999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":53.182000000000002,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.75600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9930000000000003,"5min":7.0389999999999997,"15min":2.7829999999999999},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":23.896000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":6.0339999999999998,"5min":7.4790000000000001,"15min":2.956},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":23.943000000000001,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5129999999999999,"5min":6.2370000000000001,"15min":2.8029999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.300999999999998,"5min":40.901000000000003,"15min":40.901000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":5.2530000000000001,"5min":5.7229999999999999,"15min":2.5539999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":17.434999999999999,"5min":33.162999999999997,"15min":39.137999999999998},"last":0.76600000000000001}]},{"osd":2,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.7759999999999998,"5min":6.0620000000000003,"15min":2.7469999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":18.202000000000002,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":5.8650000000000002,"5min":5.1820000000000004,"15min":2.4100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.893999999999998,"5min":25.616,"15min":58.899000000000001},"last":0.747}]},{"osd":3,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.9400000000000004,"5min":6.2759999999999998,"15min":2.6760000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":23.933,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":10.135,"5min":7.7439999999999998,"15min":3.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":34.786000000000001,"5min":111.221,"15min":111.221},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5339999999999998,"5min":5.5279999999999996,"15min":2.2120000000000002},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":20.184000000000001,"5min":51.871000000000002,"15min":51.871000000000002},"last":0.442},{"interface":"front","average":{"1min":3.6890000000000001,"5min":4.633,"15min":1.9590000000000001},"min":{"1min":0.47899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":23.858000000000001,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":4.5899999999999999,"5min":4.7770000000000001,"15min":1.9510000000000001},"min":{"1min":0.68999999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.905999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":5.673,"5min":5.524,"15min":2.2810000000000001},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":14.161,"5min":51.932000000000002,"15min":51.932000000000002},"last":0.495}]},{"osd":7,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9699999999999998,"5min":4.2699999999999996,"15min":1.8340000000000001},"min":{"1min":0.47899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":19.039000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":7.4729999999999999,"5min":5.6150000000000002,"15min":2.339},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":24.138000000000002,"5min":51.978000000000002,"15min":51.978000000000002},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.942,"5min":6.6719999999999997,"15min":2.6099999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":42.609000000000002,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":9.5069999999999997,"5min":5.5670000000000002,"15min":2.226},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":44.439,"5min":44.439,"15min":44.439},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.8230000000000004,"5min":6.2409999999999997,"15min":2.8530000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32100000000000001},"max":{"1min":31.355,"5min":32.787999999999997,"15min":124.85599999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":7.867,"5min":6,"15min":2.4079999999999999},"min":{"1min":0.48499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":31.385000000000002,"5min":41.060000000000002,"15min":41.060000000000002},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":12.061999999999999,"5min":7.4779999999999998,"15min":2.8919999999999999},"min":{"1min":0.46700000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":44.247,"5min":44.247,"15min":44.247},"last":0.69099999999999995},{"interface":"front","average":{"1min":8.5579999999999998,"5min":5.3840000000000003,"15min":2.23},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":36.612000000000002,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":19.018999999999998,"5min":7.3360000000000003,"15min":2.8769999999999998},"min":{"1min":0.49399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":79.974000000000004,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":12,"5min":10.103999999999999,"15min":3.8300000000000001},"min":{"1min":0.39900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":43.343000000000004,"5min":49.991999999999997,"15min":49.991999999999997},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.17,"5min":5.2290000000000001,"15min":2.117},"min":{"1min":0.69299999999999995,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":43.305,"5min":43.305,"15min":43.305},"last":0.40699999999999997},{"interface":"front","average":{"1min":9.3439999999999994,"5min":3.9980000000000002,"15min":1.7529999999999999},"min":{"1min":0.38600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":42.551000000000002,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.6980000000000004,"5min":7.0529999999999999,"15min":2.7629999999999999},"min":{"1min":0.53000000000000003,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":9.6489999999999991,"5min":5.4109999999999996,"15min":2.2120000000000002},"min":{"1min":0.60699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":46.505000000000003,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":8.0839999999999996,"5min":5.8360000000000003,"15min":2.4009999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":43.262,"5min":43.262,"15min":43.262},"last":0.48999999999999999},{"interface":"front","average":{"1min":9.3680000000000003,"5min":8.077,"15min":3.2120000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":44.018000000000001,"5min":80.302000000000007,"15min":80.302000000000007},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4621799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:41:13.123 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T23:41:13.123 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:41:13.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.301+0000 7f88ca561700 1 Processor -- start 2024-05-08T23:41:13.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.301+0000 7f88ca561700 1 -- start start 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.302+0000 7f88ca561700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88c40c4c70 0x7f88c415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.302+0000 7f88ca561700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f88c40d00b0 0x7f88c40d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.302+0000 7f88ca561700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c40c5c20 con 0x7f88c4154770 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.302+0000 7f88ca561700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c40b30b0 con 0x7f88c40c4c70 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.302+0000 7f88ca561700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c4154b50 con 0x7f88c40d00b0 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c3fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f88c4154770 0x7f88c40c60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40252/0 (socket says 172.21.15.33:40252) 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c3fff700 1 -- 172.21.15.33:0/3343571213 learned_addr learned my addr 172.21.15.33:0/3343571213 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f88c40c4c70 0x7f88c415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c8afe700 1 --2- 172.21.15.33:0/3343571213 >> v2:172.21.15.42:3300/0 conn(0x7f88c40d00b0 0x7f88c40d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004556753 0 0) 0x7f88c40c5c20 con 0x7f88c4154770 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b00035b0 con 0x7f88c4154770 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 -- 172.21.15.33:0/3343571213 >> v2:172.21.15.42:3300/0 conn(0x7f88c40d00b0 msgr2=0x7f88c40d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 --2- 172.21.15.33:0/3343571213 >> v2:172.21.15.42:3300/0 conn(0x7f88c40d00b0 0x7f88c40d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 -- 172.21.15.33:0/3343571213 >> v1:172.21.15.33:6789/0 conn(0x7f88c4154770 legacy=0x7f88c40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639686244 0 0) 0x7f88b00035b0 con 0x7f88c4154770 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 -- 172.21.15.33:0/3343571213 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40caed0 con 0x7f88c40c4c70 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88bbfff700 1 --2- 172.21.15.33:0/3343571213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c415ae50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88c408d8a0 tx=0x7f88ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0b26960ffbffad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4001670 con 0x7f88c40c4c70 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac004030 con 0x7f88c40c4c70 2024-05-08T23:41:13.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.303+0000 7f88c37fe700 1 -- 172.21.15.33:0/3343571213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0043a0 con 0x7f88c40c4c70 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 -- 172.21.15.33:0/3343571213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 msgr2=0x7f88c415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 --2- 172.21.15.33:0/3343571213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c415ae50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f88c408d8a0 tx=0x7f88ac009b20 comp rx=0 tx=0).stop 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 -- 172.21.15.33:0/3343571213 shutdown_connections 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 --2- 172.21.15.33:0/3343571213 >> v2:172.21.15.42:3300/0 conn(0x7f88c40d00b0 0x7f88c40d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 --2- 172.21.15.33:0/3343571213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c415ae50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 -- 172.21.15.33:0/3343571213 >> 172.21.15.33:0/3343571213 conn(0x7f88c4066ae0 msgr2=0x7f88c4066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 -- 172.21.15.33:0/3343571213 shutdown_connections 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.304+0000 7f88ca561700 1 -- 172.21.15.33:0/3343571213 wait complete. 2024-05-08T23:41:13.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.305+0000 7f88ca561700 1 Processor -- start 2024-05-08T23:41:13.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 -- start start 2024-05-08T23:41:13.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 0x7f88c414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 0x7f88c414bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c414c200 con 0x7f88c40c4c70 2024-05-08T23:41:13.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c414cc60 con 0x7f88c4154770 2024-05-08T23:41:13.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88ca561700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c414cf40 con 0x7f88c40d00b0 2024-05-08T23:41:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44354/0 (socket says 172.21.15.33:44354) 2024-05-08T23:41:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.306+0000 7f88c3fff700 1 -- 172.21.15.33:0/2367447313 learned_addr learned my addr 172.21.15.33:0/2367447313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88bbfff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 0x7f88c414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c8afe700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 0x7f88c414bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 -- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 msgr2=0x7f88c414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 0x7f88c414b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 -- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 msgr2=0x7f88c414bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 0x7f88c414bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 -- 172.21.15.33:0/2367447313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0094a0 con 0x7f88c40c4c70 2024-05-08T23:41:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c8afe700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 0x7f88c414bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88bbfff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 0x7f88c414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c3fff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c780 tx=0x7f88b400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004c80 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4004de0 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c414d220 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40072f0 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.307+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c414d7e0 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.308+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c4074640 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.309+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f88b4007450 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.309+0000 7f88c17fa700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 0x7f88a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.309+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f88b40891e0 con 0x7f88c40c4c70 2024-05-08T23:41:13.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.309+0000 7f88bbfff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 0x7f88a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.310+0000 7f88bbfff700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 0x7f88a406d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0090b0 tx=0x7f88ac008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.314+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f88c4074640 con 0x7f88c40c4c70 2024-05-08T23:41:13.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.528+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88c4073330 con 0x7f88a406ac90 2024-05-08T23:41:13.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.536+0000 7f88c17fa700 1 -- 172.21.15.33:0/2367447313 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290374 (secure 0 0 0) 0x7f88c4073330 con 0x7f88a406ac90 2024-05-08T23:41:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 msgr2=0x7f88a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 0x7f88a406d150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0090b0 tx=0x7f88ac008030 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 msgr2=0x7f88c4154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c780 tx=0x7f88b400cb40 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 shutdown_connections 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f88a406ac90 0x7f88a406d150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88c4154770 0x7f88c414bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c40c4c70 0x7f88c4154d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 --2- 172.21.15.33:0/2367447313 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f88c40d00b0 0x7f88c414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 >> 172.21.15.33:0/2367447313 conn(0x7f88c4066ae0 msgr2=0x7f88c4158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.540+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 shutdown_connections 2024-05-08T23:41:13.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.541+0000 7f88ca561700 1 -- 172.21.15.33:0/2367447313 wait complete. 2024-05-08T23:41:13.540 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:41:13.551 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-05-08T23:41:13.495244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17884029,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320948,"num_write_kb":49019246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":421939,"ondisk_log_size":421939,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4065176,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4028211,"kb_avail":750909544,"statfs":{"total":773094113280,"available":768931373056,"internally_reserved":0,"allocated":37773312,"data_stored":36472109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4124888632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003133"},"pg_stats":[{"pgid":"3.3f","version":"23'5694","reported_seq":5710,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.882557+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:40:50.882557+0000","last_peered":"2024-05-08T23:40:50.882557+0000","last_clean":"2024-05-08T23:40:50.882557+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:40:50.882557+0000","last_undegraded":"2024-05-08T23:40:50.882557+0000","last_fullsized":"2024-05-08T23:40:50.882557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8392","reported_seq":8408,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040541+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:41:12.040541+0000","last_peered":"2024-05-08T23:41:12.040541+0000","last_clean":"2024-05-08T23:41:12.040541+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:41:12.040541+0000","last_undegraded":"2024-05-08T23:41:12.040541+0000","last_fullsized":"2024-05-08T23:41:12.040541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:45:39.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6514","reported_seq":6530,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.038895+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:41:12.038895+0000","last_peered":"2024-05-08T23:41:12.038895+0000","last_clean":"2024-05-08T23:41:12.038895+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:41:12.038895+0000","last_undegraded":"2024-05-08T23:41:12.038895+0000","last_fullsized":"2024-05-08T23:41:12.038895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:41.000383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5987","reported_seq":6003,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.725504+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:40:50.725504+0000","last_peered":"2024-05-08T23:40:50.725504+0000","last_clean":"2024-05-08T23:40:50.725504+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:40:50.725504+0000","last_undegraded":"2024-05-08T23:40:50.725504+0000","last_fullsized":"2024-05-08T23:40:50.725504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5987,"log_dups_size":0,"ondisk_log_size":5987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:57.866067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7607","reported_seq":7620,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.029915+0000","last_change":"2024-05-08T23:32:59.855454+0000","last_active":"2024-05-08T23:41:12.029915+0000","last_peered":"2024-05-08T23:41:12.029915+0000","last_clean":"2024-05-08T23:41:12.029915+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:41:12.029915+0000","last_undegraded":"2024-05-08T23:41:12.029915+0000","last_fullsized":"2024-05-08T23:41:12.029915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:58.324852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5747","reported_seq":5762,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.540996+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:40:50.540996+0000","last_peered":"2024-05-08T23:40:50.540996+0000","last_clean":"2024-05-08T23:40:50.540996+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:40:50.540996+0000","last_undegraded":"2024-05-08T23:40:50.540996+0000","last_fullsized":"2024-05-08T23:40:50.540996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5747,"log_dups_size":0,"ondisk_log_size":5747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:25.432832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6492","reported_seq":6507,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.796565+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:40:50.796565+0000","last_peered":"2024-05-08T23:40:50.796565+0000","last_clean":"2024-05-08T23:40:50.796565+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:40:50.796565+0000","last_undegraded":"2024-05-08T23:40:50.796565+0000","last_fullsized":"2024-05-08T23:40:50.796565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6893","reported_seq":6908,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.108752+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:41:12.108752+0000","last_peered":"2024-05-08T23:41:12.108752+0000","last_clean":"2024-05-08T23:41:12.108752+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:41:12.108752+0000","last_undegraded":"2024-05-08T23:41:12.108752+0000","last_fullsized":"2024-05-08T23:41:12.108752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:00.284860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8081","reported_seq":8096,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.135086+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:40:50.135086+0000","last_peered":"2024-05-08T23:40:50.135086+0000","last_clean":"2024-05-08T23:40:50.135086+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:40:50.135086+0000","last_undegraded":"2024-05-08T23:40:50.135086+0000","last_fullsized":"2024-05-08T23:40:50.135086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7426","reported_seq":7441,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.158966+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:41:12.158966+0000","last_peered":"2024-05-08T23:41:12.158966+0000","last_clean":"2024-05-08T23:41:12.158966+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:41:12.158966+0000","last_undegraded":"2024-05-08T23:41:12.158966+0000","last_fullsized":"2024-05-08T23:41:12.158966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7426,"log_dups_size":0,"ondisk_log_size":7426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6322","reported_seq":6337,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.012886+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:41:12.012886+0000","last_peered":"2024-05-08T23:41:12.012886+0000","last_clean":"2024-05-08T23:41:12.012886+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:41:12.012886+0000","last_undegraded":"2024-05-08T23:41:12.012886+0000","last_fullsized":"2024-05-08T23:41:12.012886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5502","reported_seq":5517,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.801703+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:40:50.801703+0000","last_peered":"2024-05-08T23:40:50.801703+0000","last_clean":"2024-05-08T23:40:50.801703+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:40:50.801703+0000","last_undegraded":"2024-05-08T23:40:50.801703+0000","last_fullsized":"2024-05-08T23:40:50.801703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5502,"log_dups_size":0,"ondisk_log_size":5502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6039","reported_seq":6054,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.042750+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:41:12.042750+0000","last_peered":"2024-05-08T23:41:12.042750+0000","last_clean":"2024-05-08T23:41:12.042750+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:41:12.042750+0000","last_undegraded":"2024-05-08T23:41:12.042750+0000","last_fullsized":"2024-05-08T23:41:12.042750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:35:39.346204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6680","reported_seq":6695,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.060261+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:41:12.060261+0000","last_peered":"2024-05-08T23:41:12.060261+0000","last_clean":"2024-05-08T23:41:12.060261+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:41:12.060261+0000","last_undegraded":"2024-05-08T23:41:12.060261+0000","last_fullsized":"2024-05-08T23:41:12.060261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6680,"log_dups_size":0,"ondisk_log_size":6680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:00.124191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6521","reported_seq":6536,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.080624+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:41:12.080624+0000","last_peered":"2024-05-08T23:41:12.080624+0000","last_clean":"2024-05-08T23:41:12.080624+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:41:12.080624+0000","last_undegraded":"2024-05-08T23:41:12.080624+0000","last_fullsized":"2024-05-08T23:41:12.080624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6521,"log_dups_size":0,"ondisk_log_size":6521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:30.682347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5772","reported_seq":5785,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.818932+0000","last_change":"2024-05-08T23:32:59.856316+0000","last_active":"2024-05-08T23:40:50.818932+0000","last_peered":"2024-05-08T23:40:50.818932+0000","last_clean":"2024-05-08T23:40:50.818932+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:40:50.818932+0000","last_undegraded":"2024-05-08T23:40:50.818932+0000","last_fullsized":"2024-05-08T23:40:50.818932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:19.763425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7973","reported_seq":7988,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.808102+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:40:50.808102+0000","last_peered":"2024-05-08T23:40:50.808102+0000","last_clean":"2024-05-08T23:40:50.808102+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:40:50.808102+0000","last_undegraded":"2024-05-08T23:40:50.808102+0000","last_fullsized":"2024-05-08T23:40:50.808102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7973,"log_dups_size":0,"ondisk_log_size":7973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:52:25.119533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5300","reported_seq":5315,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901529+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:40:50.901529+0000","last_peered":"2024-05-08T23:40:50.901529+0000","last_clean":"2024-05-08T23:40:50.901529+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:40:50.901529+0000","last_undegraded":"2024-05-08T23:40:50.901529+0000","last_fullsized":"2024-05-08T23:40:50.901529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5300,"log_dups_size":0,"ondisk_log_size":5300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6323","reported_seq":6338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.766273+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:40:50.766273+0000","last_peered":"2024-05-08T23:40:50.766273+0000","last_clean":"2024-05-08T23:40:50.766273+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:40:50.766273+0000","last_undegraded":"2024-05-08T23:40:50.766273+0000","last_fullsized":"2024-05-08T23:40:50.766273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6323,"log_dups_size":0,"ondisk_log_size":6323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'383","reported_seq":400,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.822638+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:40:50.822638+0000","last_peered":"2024-05-08T23:40:50.822638+0000","last_clean":"2024-05-08T23:40:50.822638+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:40:50.822638+0000","last_undegraded":"2024-05-08T23:40:50.822638+0000","last_fullsized":"2024-05-08T23:40:50.822638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:31.957419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6842","reported_seq":6856,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852287+0000","last_change":"2024-05-08T23:33:00.026361+0000","last_active":"2024-05-08T23:40:50.852287+0000","last_peered":"2024-05-08T23:40:50.852287+0000","last_clean":"2024-05-08T23:40:50.852287+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:40:50.852287+0000","last_undegraded":"2024-05-08T23:40:50.852287+0000","last_fullsized":"2024-05-08T23:40:50.852287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:26.768597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828674+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:40:50.828674+0000","last_peered":"2024-05-08T23:40:50.828674+0000","last_clean":"2024-05-08T23:40:50.828674+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:40:50.828674+0000","last_undegraded":"2024-05-08T23:40:50.828674+0000","last_fullsized":"2024-05-08T23:40:50.828674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:00.716435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7917","reported_seq":7941,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.069538+0000","last_change":"2024-05-08T23:32:59.859469+0000","last_active":"2024-05-08T23:41:12.069538+0000","last_peered":"2024-05-08T23:41:12.069538+0000","last_clean":"2024-05-08T23:41:12.069538+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:41:12.069538+0000","last_undegraded":"2024-05-08T23:41:12.069538+0000","last_fullsized":"2024-05-08T23:41:12.069538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:11:35.269871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'10","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.837769+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:40:50.837769+0000","last_peered":"2024-05-08T23:40:50.837769+0000","last_clean":"2024-05-08T23:40:50.837769+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:40:50.837769+0000","last_undegraded":"2024-05-08T23:40:50.837769+0000","last_fullsized":"2024-05-08T23:40:50.837769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4519","reported_seq":4534,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.644012+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:40:50.644012+0000","last_peered":"2024-05-08T23:40:50.644012+0000","last_clean":"2024-05-08T23:40:50.644012+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:40:50.644012+0000","last_undegraded":"2024-05-08T23:40:50.644012+0000","last_fullsized":"2024-05-08T23:40:50.644012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:43:53.843044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.715531+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:40:50.715531+0000","last_peered":"2024-05-08T23:40:50.715531+0000","last_clean":"2024-05-08T23:40:50.715531+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:40:50.715531+0000","last_undegraded":"2024-05-08T23:40:50.715531+0000","last_fullsized":"2024-05-08T23:40:50.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7897","reported_seq":7912,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828353+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:40:50.828353+0000","last_peered":"2024-05-08T23:40:50.828353+0000","last_clean":"2024-05-08T23:40:50.828353+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:40:50.828353+0000","last_undegraded":"2024-05-08T23:40:50.828353+0000","last_fullsized":"2024-05-08T23:40:50.828353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:07.807303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1096","reported_seq":1112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.825119+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:40:50.825119+0000","last_peered":"2024-05-08T23:40:50.825119+0000","last_clean":"2024-05-08T23:40:50.825119+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:40:50.825119+0000","last_undegraded":"2024-05-08T23:40:50.825119+0000","last_fullsized":"2024-05-08T23:40:50.825119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5444","reported_seq":5459,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.736987+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:40:50.736987+0000","last_peered":"2024-05-08T23:40:50.736987+0000","last_clean":"2024-05-08T23:40:50.736987+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:40:50.736987+0000","last_undegraded":"2024-05-08T23:40:50.736987+0000","last_fullsized":"2024-05-08T23:40:50.736987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5444,"log_dups_size":0,"ondisk_log_size":5444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'26","reported_seq":42,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.456503+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:40:51.456503+0000","last_peered":"2024-05-08T23:40:51.456503+0000","last_clean":"2024-05-08T23:40:51.456503+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:40:51.456503+0000","last_undegraded":"2024-05-08T23:40:51.456503+0000","last_fullsized":"2024-05-08T23:40:51.456503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426996,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5065","reported_seq":5080,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.136189+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:41:12.136189+0000","last_peered":"2024-05-08T23:41:12.136189+0000","last_clean":"2024-05-08T23:41:12.136189+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:41:12.136189+0000","last_undegraded":"2024-05-08T23:41:12.136189+0000","last_fullsized":"2024-05-08T23:41:12.136189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:00:56.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836686+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:40:50.836686+0000","last_peered":"2024-05-08T23:40:50.836686+0000","last_clean":"2024-05-08T23:40:50.836686+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:40:50.836686+0000","last_undegraded":"2024-05-08T23:40:50.836686+0000","last_fullsized":"2024-05-08T23:40:50.836686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:18.562950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6593","reported_seq":6606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901627+0000","last_change":"2024-05-08T23:32:59.856036+0000","last_active":"2024-05-08T23:40:50.901627+0000","last_peered":"2024-05-08T23:40:50.901627+0000","last_clean":"2024-05-08T23:40:50.901627+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:40:50.901627+0000","last_undegraded":"2024-05-08T23:40:50.901627+0000","last_fullsized":"2024-05-08T23:40:50.901627+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6593,"log_dups_size":0,"ondisk_log_size":6593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:07.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:37:06.070170+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:37:06.070170+0000","last_peered":"2024-05-08T23:37:06.070170+0000","last_clean":"2024-05-08T23:37:06.070170+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:37:06.070170+0000","last_undegraded":"2024-05-08T23:37:06.070170+0000","last_fullsized":"2024-05-08T23:37:06.070170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:27.977110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7591","reported_seq":7606,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.066970+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:41:12.066970+0000","last_peered":"2024-05-08T23:41:12.066970+0000","last_clean":"2024-05-08T23:41:12.066970+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:41:12.066970+0000","last_undegraded":"2024-05-08T23:41:12.066970+0000","last_fullsized":"2024-05-08T23:41:12.066970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:20.785791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.812887+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:40:50.812887+0000","last_peered":"2024-05-08T23:40:50.812887+0000","last_clean":"2024-05-08T23:40:50.812887+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:40:50.812887+0000","last_undegraded":"2024-05-08T23:40:50.812887+0000","last_fullsized":"2024-05-08T23:40:50.812887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6493","reported_seq":6508,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885950+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:40:50.885950+0000","last_peered":"2024-05-08T23:40:50.885950+0000","last_clean":"2024-05-08T23:40:50.885950+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:40:50.885950+0000","last_undegraded":"2024-05-08T23:40:50.885950+0000","last_fullsized":"2024-05-08T23:40:50.885950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6493,"log_dups_size":0,"ondisk_log_size":6493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:08.078950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.804197+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:40:50.804197+0000","last_peered":"2024-05-08T23:40:50.804197+0000","last_clean":"2024-05-08T23:40:50.804197+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:40:50.804197+0000","last_undegraded":"2024-05-08T23:40:50.804197+0000","last_fullsized":"2024-05-08T23:40:50.804197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:01:07.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5821","reported_seq":5844,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.061111+0000","last_change":"2024-05-08T23:32:59.859486+0000","last_active":"2024-05-08T23:41:12.061111+0000","last_peered":"2024-05-08T23:41:12.061111+0000","last_clean":"2024-05-08T23:41:12.061111+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:41:12.061111+0000","last_undegraded":"2024-05-08T23:41:12.061111+0000","last_fullsized":"2024-05-08T23:41:12.061111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:23.076314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'17","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869954+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:40:50.869954+0000","last_peered":"2024-05-08T23:40:50.869954+0000","last_clean":"2024-05-08T23:40:50.869954+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:40:50.869954+0000","last_undegraded":"2024-05-08T23:40:50.869954+0000","last_fullsized":"2024-05-08T23:40:50.869954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:00.360683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8441","reported_seq":8456,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.885980+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:40:50.885980+0000","last_peered":"2024-05-08T23:40:50.885980+0000","last_clean":"2024-05-08T23:40:50.885980+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:40:50.885980+0000","last_undegraded":"2024-05-08T23:40:50.885980+0000","last_fullsized":"2024-05-08T23:40:50.885980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8441,"log_dups_size":0,"ondisk_log_size":8441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'223","reported_seq":239,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.718908+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:40:50.718908+0000","last_peered":"2024-05-08T23:40:50.718908+0000","last_clean":"2024-05-08T23:40:50.718908+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:40:50.718908+0000","last_undegraded":"2024-05-08T23:40:50.718908+0000","last_fullsized":"2024-05-08T23:40:50.718908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6150","reported_seq":6165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.633129+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:40:50.633129+0000","last_peered":"2024-05-08T23:40:50.633129+0000","last_clean":"2024-05-08T23:40:50.633129+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:40:50.633129+0000","last_undegraded":"2024-05-08T23:40:50.633129+0000","last_fullsized":"2024-05-08T23:40:50.633129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6150,"log_dups_size":0,"ondisk_log_size":6150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854210+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:40:50.854210+0000","last_peered":"2024-05-08T23:40:50.854210+0000","last_clean":"2024-05-08T23:40:50.854210+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:40:50.854210+0000","last_undegraded":"2024-05-08T23:40:50.854210+0000","last_fullsized":"2024-05-08T23:40:50.854210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:50.725013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5984","reported_seq":5999,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.078710+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:41:12.078710+0000","last_peered":"2024-05-08T23:41:12.078710+0000","last_clean":"2024-05-08T23:41:12.078710+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:41:12.078710+0000","last_undegraded":"2024-05-08T23:41:12.078710+0000","last_fullsized":"2024-05-08T23:41:12.078710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:04.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.883912+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:40:50.883912+0000","last_peered":"2024-05-08T23:40:50.883912+0000","last_clean":"2024-05-08T23:40:50.883912+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:40:50.883912+0000","last_undegraded":"2024-05-08T23:40:50.883912+0000","last_fullsized":"2024-05-08T23:40:50.883912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:50.408215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7310","reported_seq":7325,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.859323+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:40:50.859323+0000","last_peered":"2024-05-08T23:40:50.859323+0000","last_clean":"2024-05-08T23:40:50.859323+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:40:50.859323+0000","last_undegraded":"2024-05-08T23:40:50.859323+0000","last_fullsized":"2024-05-08T23:40:50.859323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7310,"log_dups_size":0,"ondisk_log_size":7310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.831450+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:40:50.831450+0000","last_peered":"2024-05-08T23:40:50.831450+0000","last_clean":"2024-05-08T23:40:50.831450+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:40:50.831450+0000","last_undegraded":"2024-05-08T23:40:50.831450+0000","last_fullsized":"2024-05-08T23:40:50.831450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7017","reported_seq":7032,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.069071+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:41:12.069071+0000","last_peered":"2024-05-08T23:41:12.069071+0000","last_clean":"2024-05-08T23:41:12.069071+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:41:12.069071+0000","last_undegraded":"2024-05-08T23:41:12.069071+0000","last_fullsized":"2024-05-08T23:41:12.069071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7017,"log_dups_size":0,"ondisk_log_size":7017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'15","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828797+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:40:50.828797+0000","last_peered":"2024-05-08T23:40:50.828797+0000","last_clean":"2024-05-08T23:40:50.828797+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:40:50.828797+0000","last_undegraded":"2024-05-08T23:40:50.828797+0000","last_fullsized":"2024-05-08T23:40:50.828797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6605","reported_seq":6620,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.792042+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:40:50.792042+0000","last_peered":"2024-05-08T23:40:50.792042+0000","last_clean":"2024-05-08T23:40:50.792042+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:40:50.792042+0000","last_undegraded":"2024-05-08T23:40:50.792042+0000","last_fullsized":"2024-05-08T23:40:50.792042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6605,"log_dups_size":0,"ondisk_log_size":6605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836668+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:40:50.836668+0000","last_peered":"2024-05-08T23:40:50.836668+0000","last_clean":"2024-05-08T23:40:50.836668+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:40:50.836668+0000","last_undegraded":"2024-05-08T23:40:50.836668+0000","last_fullsized":"2024-05-08T23:40:50.836668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6071","reported_seq":6094,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870424+0000","last_change":"2024-05-08T23:33:00.026523+0000","last_active":"2024-05-08T23:40:50.870424+0000","last_peered":"2024-05-08T23:40:50.870424+0000","last_clean":"2024-05-08T23:40:50.870424+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:40:50.870424+0000","last_undegraded":"2024-05-08T23:40:50.870424+0000","last_fullsized":"2024-05-08T23:40:50.870424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6071,"log_dups_size":0,"ondisk_log_size":6071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:23.920007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826077+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:40:50.826077+0000","last_peered":"2024-05-08T23:40:50.826077+0000","last_clean":"2024-05-08T23:40:50.826077+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:40:50.826077+0000","last_undegraded":"2024-05-08T23:40:50.826077+0000","last_fullsized":"2024-05-08T23:40:50.826077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:15:20.368445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5809","reported_seq":5825,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.854937+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:40:50.854937+0000","last_peered":"2024-05-08T23:40:50.854937+0000","last_clean":"2024-05-08T23:40:50.854937+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:40:50.854937+0000","last_undegraded":"2024-05-08T23:40:50.854937+0000","last_fullsized":"2024-05-08T23:40:50.854937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5809,"log_dups_size":0,"ondisk_log_size":5809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:10.359937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:27.185535+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:38:27.185535+0000","last_peered":"2024-05-08T23:38:27.185535+0000","last_clean":"2024-05-08T23:38:27.185535+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:38:27.185535+0000","last_undegraded":"2024-05-08T23:38:27.185535+0000","last_fullsized":"2024-05-08T23:38:27.185535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:07:28.834332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6329","reported_seq":6345,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.684184+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:40:50.684184+0000","last_peered":"2024-05-08T23:40:50.684184+0000","last_clean":"2024-05-08T23:40:50.684184+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:40:50.684184+0000","last_undegraded":"2024-05-08T23:40:50.684184+0000","last_fullsized":"2024-05-08T23:40:50.684184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6281","reported_seq":6297,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.063293+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:41:12.063293+0000","last_peered":"2024-05-08T23:41:12.063293+0000","last_clean":"2024-05-08T23:41:12.063293+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:41:12.063293+0000","last_undegraded":"2024-05-08T23:41:12.063293+0000","last_fullsized":"2024-05-08T23:41:12.063293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:45.844294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'27","reported_seq":68,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828876+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:40:50.828876+0000","last_peered":"2024-05-08T23:40:50.828876+0000","last_clean":"2024-05-08T23:40:50.828876+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:40:50.828876+0000","last_undegraded":"2024-05-08T23:40:50.828876+0000","last_fullsized":"2024-05-08T23:40:50.828876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6855","reported_seq":6871,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.095989+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:41:12.095989+0000","last_peered":"2024-05-08T23:41:12.095989+0000","last_clean":"2024-05-08T23:41:12.095989+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:41:12.095989+0000","last_undegraded":"2024-05-08T23:41:12.095989+0000","last_fullsized":"2024-05-08T23:41:12.095989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:08.572663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.805976+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:40:50.805976+0000","last_peered":"2024-05-08T23:40:50.805976+0000","last_clean":"2024-05-08T23:40:50.805976+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:40:50.805976+0000","last_undegraded":"2024-05-08T23:40:50.805976+0000","last_fullsized":"2024-05-08T23:40:50.805976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:23.313078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:16.309946+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:38:16.309946+0000","last_peered":"2024-05-08T23:38:16.309946+0000","last_clean":"2024-05-08T23:38:16.309946+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:38:16.309946+0000","last_undegraded":"2024-05-08T23:38:16.309946+0000","last_fullsized":"2024-05-08T23:38:16.309946+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6396","reported_seq":6412,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.068930+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:41:12.068930+0000","last_peered":"2024-05-08T23:41:12.068930+0000","last_clean":"2024-05-08T23:41:12.068930+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:41:12.068930+0000","last_undegraded":"2024-05-08T23:41:12.068930+0000","last_fullsized":"2024-05-08T23:41:12.068930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6396,"log_dups_size":0,"ondisk_log_size":6396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:12.415698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849138+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:40:50.849138+0000","last_peered":"2024-05-08T23:40:50.849138+0000","last_clean":"2024-05-08T23:40:50.849138+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:40:50.849138+0000","last_undegraded":"2024-05-08T23:40:50.849138+0000","last_fullsized":"2024-05-08T23:40:50.849138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:38.734890+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:40:38.734890+0000","last_peered":"2024-05-08T23:40:38.734890+0000","last_clean":"2024-05-08T23:40:38.734890+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:40:38.734890+0000","last_undegraded":"2024-05-08T23:40:38.734890+0000","last_fullsized":"2024-05-08T23:40:38.734890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7015","reported_seq":7031,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.034830+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:41:12.034830+0000","last_peered":"2024-05-08T23:41:12.034830+0000","last_clean":"2024-05-08T23:41:12.034830+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:41:12.034830+0000","last_undegraded":"2024-05-08T23:41:12.034830+0000","last_fullsized":"2024-05-08T23:41:12.034830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:32.317542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'86","reported_seq":103,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.826703+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:40:50.826703+0000","last_peered":"2024-05-08T23:40:50.826703+0000","last_clean":"2024-05-08T23:40:50.826703+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:40:50.826703+0000","last_undegraded":"2024-05-08T23:40:50.826703+0000","last_fullsized":"2024-05-08T23:40:50.826703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:11:38.481157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":200,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6431","reported_seq":6447,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.000935+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:41:12.000935+0000","last_peered":"2024-05-08T23:41:12.000935+0000","last_clean":"2024-05-08T23:41:12.000935+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:41:12.000935+0000","last_undegraded":"2024-05-08T23:41:12.000935+0000","last_fullsized":"2024-05-08T23:41:12.000935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:27.722678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.423772+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:40:51.423772+0000","last_peered":"2024-05-08T23:40:51.423772+0000","last_clean":"2024-05-08T23:40:51.423772+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:40:51.423772+0000","last_undegraded":"2024-05-08T23:40:51.423772+0000","last_fullsized":"2024-05-08T23:40:51.423772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5458","reported_seq":5474,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040664+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:41:12.040664+0000","last_peered":"2024-05-08T23:41:12.040664+0000","last_clean":"2024-05-08T23:41:12.040664+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:41:12.040664+0000","last_undegraded":"2024-05-08T23:41:12.040664+0000","last_fullsized":"2024-05-08T23:41:12.040664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:47.917322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.851008+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:40:50.851008+0000","last_peered":"2024-05-08T23:40:50.851008+0000","last_clean":"2024-05-08T23:40:50.851008+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:40:50.851008+0000","last_undegraded":"2024-05-08T23:40:50.851008+0000","last_fullsized":"2024-05-08T23:40:50.851008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5843","reported_seq":5859,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870876+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:40:50.870876+0000","last_peered":"2024-05-08T23:40:50.870876+0000","last_clean":"2024-05-08T23:40:50.870876+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:40:50.870876+0000","last_undegraded":"2024-05-08T23:40:50.870876+0000","last_fullsized":"2024-05-08T23:40:50.870876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5843,"log_dups_size":0,"ondisk_log_size":5843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.835933+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:40:50.835933+0000","last_peered":"2024-05-08T23:40:50.835933+0000","last_clean":"2024-05-08T23:40:50.835933+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:40:50.835933+0000","last_undegraded":"2024-05-08T23:40:50.835933+0000","last_fullsized":"2024-05-08T23:40:50.835933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:59.617802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6488","reported_seq":6504,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.041903+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:41:12.041903+0000","last_peered":"2024-05-08T23:41:12.041903+0000","last_clean":"2024-05-08T23:41:12.041903+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:41:12.041903+0000","last_undegraded":"2024-05-08T23:41:12.041903+0000","last_fullsized":"2024-05-08T23:41:12.041903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:51.650166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.875794+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:40:50.875794+0000","last_peered":"2024-05-08T23:40:50.875794+0000","last_clean":"2024-05-08T23:40:50.875794+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:40:50.875794+0000","last_undegraded":"2024-05-08T23:40:50.875794+0000","last_fullsized":"2024-05-08T23:40:50.875794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:05.339267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5341","reported_seq":5357,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.067701+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:41:12.067701+0000","last_peered":"2024-05-08T23:41:12.067701+0000","last_clean":"2024-05-08T23:41:12.067701+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:41:12.067701+0000","last_undegraded":"2024-05-08T23:41:12.067701+0000","last_fullsized":"2024-05-08T23:41:12.067701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:44:25.973365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'685","reported_seq":701,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852470+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:40:50.852470+0000","last_peered":"2024-05-08T23:40:50.852470+0000","last_clean":"2024-05-08T23:40:50.852470+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:40:50.852470+0000","last_undegraded":"2024-05-08T23:40:50.852470+0000","last_fullsized":"2024-05-08T23:40:50.852470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5938","reported_seq":5954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051117+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:41:12.051117+0000","last_peered":"2024-05-08T23:41:12.051117+0000","last_clean":"2024-05-08T23:41:12.051117+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:41:12.051117+0000","last_undegraded":"2024-05-08T23:41:12.051117+0000","last_fullsized":"2024-05-08T23:41:12.051117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.896708+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:40:50.896708+0000","last_peered":"2024-05-08T23:40:50.896708+0000","last_clean":"2024-05-08T23:40:50.896708+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:40:50.896708+0000","last_undegraded":"2024-05-08T23:40:50.896708+0000","last_fullsized":"2024-05-08T23:40:50.896708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6537","reported_seq":6552,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.899748+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:40:50.899748+0000","last_peered":"2024-05-08T23:40:50.899748+0000","last_clean":"2024-05-08T23:40:50.899748+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:40:50.899748+0000","last_undegraded":"2024-05-08T23:40:50.899748+0000","last_fullsized":"2024-05-08T23:40:50.899748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6537,"log_dups_size":0,"ondisk_log_size":6537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:39:01.910099+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:39:01.910099+0000","last_peered":"2024-05-08T23:39:01.910099+0000","last_clean":"2024-05-08T23:39:01.910099+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:39:01.910099+0000","last_undegraded":"2024-05-08T23:39:01.910099+0000","last_fullsized":"2024-05-08T23:39:01.910099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.383200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6818","reported_seq":6833,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.897583+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:40:50.897583+0000","last_peered":"2024-05-08T23:40:50.897583+0000","last_clean":"2024-05-08T23:40:50.897583+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:40:50.897583+0000","last_undegraded":"2024-05-08T23:40:50.897583+0000","last_fullsized":"2024-05-08T23:40:50.897583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6818,"log_dups_size":0,"ondisk_log_size":6818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:36.385091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828990+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:40:50.828990+0000","last_peered":"2024-05-08T23:40:50.828990+0000","last_clean":"2024-05-08T23:40:50.828990+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:40:50.828990+0000","last_undegraded":"2024-05-08T23:40:50.828990+0000","last_fullsized":"2024-05-08T23:40:50.828990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7691","reported_seq":7706,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.671715+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:40:50.671715+0000","last_peered":"2024-05-08T23:40:50.671715+0000","last_clean":"2024-05-08T23:40:50.671715+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:40:50.671715+0000","last_undegraded":"2024-05-08T23:40:50.671715+0000","last_fullsized":"2024-05-08T23:40:50.671715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7806","reported_seq":7822,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.080477+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:41:12.080477+0000","last_peered":"2024-05-08T23:41:12.080477+0000","last_clean":"2024-05-08T23:41:12.080477+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:41:12.080477+0000","last_undegraded":"2024-05-08T23:41:12.080477+0000","last_fullsized":"2024-05-08T23:41:12.080477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7806,"log_dups_size":0,"ondisk_log_size":7806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6562","reported_seq":6578,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.602602+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:40:50.602602+0000","last_peered":"2024-05-08T23:40:50.602602+0000","last_clean":"2024-05-08T23:40:50.602602+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:40:50.602602+0000","last_undegraded":"2024-05-08T23:40:50.602602+0000","last_fullsized":"2024-05-08T23:40:50.602602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6562,"log_dups_size":0,"ondisk_log_size":6562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8077","reported_seq":8093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.884270+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:40:50.884270+0000","last_peered":"2024-05-08T23:40:50.884270+0000","last_clean":"2024-05-08T23:40:50.884270+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:40:50.884270+0000","last_undegraded":"2024-05-08T23:40:50.884270+0000","last_fullsized":"2024-05-08T23:40:50.884270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:30.022581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5657","reported_seq":5673,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.097043+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:41:12.097043+0000","last_peered":"2024-05-08T23:41:12.097043+0000","last_clean":"2024-05-08T23:41:12.097043+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:41:12.097043+0000","last_undegraded":"2024-05-08T23:41:12.097043+0000","last_fullsized":"2024-05-08T23:41:12.097043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:34.488915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6017","reported_seq":6033,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.072971+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:41:12.072971+0000","last_peered":"2024-05-08T23:41:12.072971+0000","last_clean":"2024-05-08T23:41:12.072971+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:41:12.072971+0000","last_undegraded":"2024-05-08T23:41:12.072971+0000","last_fullsized":"2024-05-08T23:41:12.072971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:20:44.276461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7053","reported_seq":7069,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.009642+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:41:12.009642+0000","last_peered":"2024-05-08T23:41:12.009642+0000","last_clean":"2024-05-08T23:41:12.009642+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:41:12.009642+0000","last_undegraded":"2024-05-08T23:41:12.009642+0000","last_fullsized":"2024-05-08T23:41:12.009642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:43.390097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5666","reported_seq":5682,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.152832+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:41:12.152832+0000","last_peered":"2024-05-08T23:41:12.152832+0000","last_clean":"2024-05-08T23:41:12.152832+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:41:12.152832+0000","last_undegraded":"2024-05-08T23:41:12.152832+0000","last_fullsized":"2024-05-08T23:41:12.152832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7698","reported_seq":7714,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:11.975050+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:41:11.975050+0000","last_peered":"2024-05-08T23:41:11.975050+0000","last_clean":"2024-05-08T23:41:11.975050+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:41:11.975050+0000","last_undegraded":"2024-05-08T23:41:11.975050+0000","last_fullsized":"2024-05-08T23:41:11.975050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:43.718747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5551","reported_seq":5567,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.869705+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:40:50.869705+0000","last_peered":"2024-05-08T23:40:50.869705+0000","last_clean":"2024-05-08T23:40:50.869705+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:40:50.869705+0000","last_undegraded":"2024-05-08T23:40:50.869705+0000","last_fullsized":"2024-05-08T23:40:50.869705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5551,"log_dups_size":0,"ondisk_log_size":5551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:37.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6393","reported_seq":6409,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.072641+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:41:12.072641+0000","last_peered":"2024-05-08T23:41:12.072641+0000","last_clean":"2024-05-08T23:41:12.072641+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:41:12.072641+0000","last_undegraded":"2024-05-08T23:41:12.072641+0000","last_fullsized":"2024-05-08T23:41:12.072641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6393,"log_dups_size":0,"ondisk_log_size":6393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:46:28.873507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7231","reported_seq":7246,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.609971+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:40:50.609971+0000","last_peered":"2024-05-08T23:40:50.609971+0000","last_clean":"2024-05-08T23:40:50.609971+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:40:50.609971+0000","last_undegraded":"2024-05-08T23:40:50.609971+0000","last_fullsized":"2024-05-08T23:40:50.609971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7231,"log_dups_size":0,"ondisk_log_size":7231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:56.127789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6810","reported_seq":6825,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.155010+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:41:12.155010+0000","last_peered":"2024-05-08T23:41:12.155010+0000","last_clean":"2024-05-08T23:41:12.155010+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:41:12.155010+0000","last_undegraded":"2024-05-08T23:41:12.155010+0000","last_fullsized":"2024-05-08T23:41:12.155010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:00:39.433207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6394","reported_seq":6409,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.083630+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:41:12.083630+0000","last_peered":"2024-05-08T23:41:12.083630+0000","last_clean":"2024-05-08T23:41:12.083630+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:41:12.083630+0000","last_undegraded":"2024-05-08T23:41:12.083630+0000","last_fullsized":"2024-05-08T23:41:12.083630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419170,"ondisk_log_size":419170,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293661,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2680,"num_write_kb":19123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34587149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2652,"ondisk_log_size":2652,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633788,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":620670,"kb_avail":93738052,"statfs":{"total":96636764160,"available":95987765248,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":635566539},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":8.4469999999999992,"15min":3.2210000000000001},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":45.125999999999998,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":9.0259999999999998,"5min":7.827,"15min":3.0419999999999998},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":37.293999999999997,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.194,"5min":4.4950000000000001,"15min":1.9470000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":13.531000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":9.1940000000000008,"5min":7.9210000000000003,"15min":3.1160000000000001},"min":{"1min":0.40000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":24.387,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.5529999999999999,"5min":4.2610000000000001,"15min":1.8879999999999999},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":15.938000000000001,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":6.0819999999999999,"5min":5.0229999999999997,"15min":2.1219999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":19.565000000000001,"5min":26.084,"15min":26.084},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":6.0049999999999999,"5min":7.1989999999999998,"15min":2.8839999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":19.614000000000001,"5min":52.869,"15min":52.869},"last":0.52900000000000003},{"interface":"front","average":{"1min":10.468,"5min":7.2720000000000002,"15min":2.855},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35499999999999998},"max":{"1min":49.421999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.67500000000000004}]},{"osd":4,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":5.0599999999999996,"15min":2.0430000000000001},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.9139999999999997,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":5.3949999999999996,"5min":3.8919999999999999,"15min":1.6950000000000001},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":22.901,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.495}]},{"osd":5,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":7.1719999999999997,"5min":5.1349999999999998,"15min":2.0910000000000002},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":18.988,"5min":63.732999999999997,"15min":63.732999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":9.2859999999999996,"5min":7.7160000000000002,"15min":2.9860000000000002},"min":{"1min":0.69199999999999995,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":18.504999999999999,"5min":62.558,"15min":62.558},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":10.367000000000001,"5min":7.5259999999999998,"15min":2.9529999999999998},"min":{"1min":0.71799999999999997,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":45.826999999999998,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.56599999999999995},{"interface":"front","average":{"1min":5.8150000000000004,"5min":4.6600000000000001,"15min":1.982},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40600000000000003},"max":{"1min":38.801000000000002,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.60499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4710000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.7320000000000002,"5min":6.8529999999999998,"15min":2.7749999999999999},"min":{"1min":0.52500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":19.369,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":6.4900000000000002,"5min":7.8170000000000002,"15min":3.1259999999999999},"min":{"1min":0.47199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":18.379000000000001,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":6.8890000000000002,"5min":6.3970000000000002,"15min":2.8570000000000002},"min":{"1min":0.876,"5min":0.313,"15min":0.313},"max":{"1min":24.693000000000001,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":5.6459999999999999,"5min":5.7210000000000001,"15min":2.6230000000000002},"min":{"1min":0.57899999999999996,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":24.187999999999999,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":4.306,"5min":4.7060000000000004,"15min":2.1030000000000002},"min":{"1min":0.58499999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":15.589,"5min":41.445,"15min":41.445},"last":0.48599999999999999},{"interface":"front","average":{"1min":4.3399999999999999,"5min":4.6550000000000002,"15min":2.0630000000000002},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":15.624000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.4909999999999997,"5min":5.2729999999999997,"15min":2.266},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":15.467000000000001,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":5.8230000000000004,"5min":7.4550000000000001,"15min":3.1749999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":28.600000000000001,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":3.141,"15min":1.446},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.891,"5min":20.913,"15min":20.913},"last":0.41599999999999998},{"interface":"front","average":{"1min":2.1909999999999998,"5min":3.875,"15min":1.8129999999999999},"min":{"1min":0.52700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.9569999999999999,"5min":35.975999999999999,"15min":35.975999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":7.3280000000000003,"5min":6.5369999999999999,"15min":2.657},"min":{"1min":0.64500000000000002,"5min":0.39200000000000002,"15min":0.33100000000000002},"max":{"1min":36.756,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":6.0759999999999996,"5min":4.633,"15min":2.0230000000000001},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":23.105,"5min":23.105,"15min":23.105},"last":0.39200000000000002}]},{"osd":7,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":4.2359999999999998,"15min":2.0259999999999998},"min":{"1min":0.44800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":6.9900000000000002,"5min":24.16,"15min":33.463000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":5.7530000000000001,"5min":5.6429999999999998,"15min":2.3460000000000001},"min":{"1min":0.83299999999999996,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.056999999999999,"5min":45.805,"15min":45.805},"last":0.46500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":7.673,"5min":6.6539999999999999,"15min":2.5219999999999998},"min":{"1min":0.437,"5min":0.29099999999999998,"15min":0.23400000000000001},"max":{"1min":30.721,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":4.6159999999999997,"5min":5.1420000000000003,"15min":2.1499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":13.93,"5min":48.844999999999999,"15min":48.844999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.6609999999999996,"5min":4.5510000000000002,"15min":1.897},"min":{"1min":0.32400000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":22.721,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.5819999999999999,"15min":2.3359999999999999},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":25.683,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.44}]},{"osd":3,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":7.8360000000000003,"15min":2.9780000000000002},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":21.975000000000001,"5min":52.712000000000003,"15min":52.712000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":4.9900000000000002,"5min":9.1709999999999994,"15min":3.4780000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.32100000000000001},"max":{"1min":25.724,"5min":61.832000000000001,"15min":61.832000000000001},"last":0.495}]},{"osd":4,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":6.2939999999999996,"5min":6.4749999999999996,"15min":2.6850000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":23.529,"5min":45.267000000000003,"15min":45.267000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":7.2350000000000003,"5min":6.3070000000000004,"15min":2.5979999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":31.245999999999999,"5min":43.353999999999999,"15min":43.353999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":5.3419999999999996,"5min":6.3719999999999999,"15min":2.6850000000000001},"min":{"1min":0.66300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":16.907,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":6.984,"5min":8.1400000000000006,"15min":3.2719999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":36.280999999999999,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.68100000000000005}]},{"osd":6,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.4470000000000001,"5min":7.5030000000000001,"15min":3.0939999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":11.292999999999999,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":6.8490000000000002,"5min":7.016,"15min":2.8900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":25.600999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":10.384,"5min":8.8420000000000005,"15min":3.4609999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":72.088999999999999,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.63},{"interface":"front","average":{"1min":10.420999999999999,"5min":9.4009999999999998,"15min":3.7349999999999999},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":72.150000000000006,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.52800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.324,"5min":6.6020000000000003,"15min":2.8780000000000001},"min":{"1min":0.28599999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":47.241999999999997,"5min":47.241999999999997,"15min":101.899},"last":0.316},{"interface":"front","average":{"1min":7.3029999999999999,"5min":5.3879999999999999,"15min":2.5339999999999998},"min":{"1min":0.65000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":22.170999999999999,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.34000000000000002}]},{"osd":2,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":14.221,"5min":8.1920000000000002,"15min":3.5609999999999999},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":134.16300000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.378},{"interface":"front","average":{"1min":10.577999999999999,"5min":7.7560000000000002,"15min":3.3380000000000001},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":47.277000000000001,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.8740000000000006,"5min":7.0720000000000001,"15min":3.1829999999999998},"min":{"1min":0.59399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":59.796999999999997,"5min":59.796999999999997,"15min":101.929},"last":0.42999999999999999},{"interface":"front","average":{"1min":4.3179999999999996,"5min":4.8780000000000001,"15min":2.2130000000000001},"min":{"1min":0.47499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.606,"5min":95.200999999999993,"15min":95.200999999999993},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.207000000000001,"5min":6.8620000000000001,"15min":3.206},"min":{"1min":0.79500000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":47.194000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.68300000000000005},{"interface":"front","average":{"1min":4.2290000000000001,"5min":5.282,"15min":2.3500000000000001},"min":{"1min":0.54300000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":12.694000000000001,"5min":39.813000000000002,"15min":44.545999999999999},"last":0.621}]},{"osd":5,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.326000000000001,"5min":6.3300000000000001,"15min":2.8500000000000001},"min":{"1min":1.111,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":38.494,"5min":38.494,"15min":78.780000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":5.0940000000000003,"5min":6.5419999999999998,"15min":2.7690000000000001},"min":{"1min":0.79800000000000004,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":12.657999999999999,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.58299999999999996}]},{"osd":6,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.0920000000000005,"5min":7.1189999999999998,"15min":3.2959999999999998},"min":{"1min":0.85299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":45.744999999999997,"5min":74.013000000000005,"15min":101.869},"last":0.47099999999999997},{"interface":"front","average":{"1min":10.259,"5min":6.2560000000000002,"15min":2.903},"min":{"1min":0.46100000000000002,"5min":0.317,"15min":0.317},"max":{"1min":38.545000000000002,"5min":71.323999999999998,"15min":78.647999999999996},"last":0.64600000000000002}]},{"osd":7,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.394,"5min":6,"15min":2.8479999999999999},"min":{"1min":0.67200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":37.725000000000001,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.9059999999999997,"15min":2.2759999999999998},"min":{"1min":0.61199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":13.746,"5min":27.056000000000001,"15min":44.713999999999999},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574958,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":5.6520000000000001,"5min":5.6859999999999999,"15min":2.1960000000000002},"min":{"1min":0.35499999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":44.238999999999997,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":6.6989999999999998,"5min":5.024,"15min":2.0939999999999999},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.27700000000000002},"max":{"1min":44.353000000000002,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":10.422000000000001,"5min":6.5999999999999996,"15min":2.5630000000000002},"min":{"1min":0.29199999999999998,"5min":0.28100000000000003,"15min":0.255},"max":{"1min":79.540999999999997,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":11.664,"5min":6.3029999999999999,"15min":2.476},"min":{"1min":0.52100000000000002,"5min":0.374,"15min":0.318},"max":{"1min":79.584000000000003,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":8.9100000000000001,"5min":9.1769999999999996,"15min":3.3730000000000002},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.27600000000000002},"max":{"1min":26.959,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.40300000000000002},{"interface":"front","average":{"1min":3.363,"5min":5.0359999999999996,"15min":2.0699999999999998},"min":{"1min":0.371,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":18.030999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.4589999999999996,"5min":5.0419999999999998,"15min":2.169},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":41.488999999999997,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":4.4950000000000001,"5min":4.5179999999999998,"15min":1.9850000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":17.946999999999999,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.4720000000000004,"5min":6.577,"15min":2.7519999999999998},"min":{"1min":0.88700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":24.843,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":5.4720000000000004,"5min":5.3280000000000003,"15min":2.2970000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":14.734999999999999,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":6.2080000000000002,"5min":5.0599999999999996,"15min":2.1970000000000001},"min":{"1min":0.47599999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":41.396999999999998,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":7.5640000000000001,"5min":6.2050000000000001,"15min":2.5430000000000001},"min":{"1min":0.93999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":24.686,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Wed May 8 23:40:15 2024","interfaces":[{"interface":"back","average":{"1min":7.2050000000000001,"5min":5.2779999999999996,"15min":2.3100000000000001},"min":{"1min":0.70299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":24.623999999999999,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":4.6799999999999997,"5min":7.0670000000000002,"15min":2.8570000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":17.905000000000001,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.70999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":4.125,"5min":6.7089999999999996,"15min":2.5619999999999998},"min":{"1min":0.35299999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":18.997,"5min":67.305999999999997,"15min":67.305999999999997},"last":5.8979999999999997},{"interface":"front","average":{"1min":6.2910000000000004,"5min":9.1940000000000008,"15min":3.3969999999999998},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.149999999999999,"5min":106.934,"15min":106.934},"last":0.42899999999999999}]},{"osd":1,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":9.6839999999999993,"5min":7.0439999999999996,"15min":2.6669999999999998},"min":{"1min":0.40699999999999997,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":76.628,"5min":76.628,"15min":76.628},"last":1.5600000000000001},{"interface":"front","average":{"1min":4.9690000000000003,"5min":6.4690000000000003,"15min":2.5129999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":19.045999999999999,"5min":54.991999999999997,"15min":54.991999999999997},"last":1.522}]},{"osd":2,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":8.5600000000000005,"5min":7.0899999999999999,"15min":3.008},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":57.378,"5min":57.378,"15min":57.378},"last":65.519000000000005},{"interface":"front","average":{"1min":4.3460000000000001,"5min":5.8959999999999999,"15min":2.363},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":19.800000000000001,"5min":44.951000000000001,"15min":44.951000000000001},"last":5.5659999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":11.138999999999999,"5min":8.3179999999999996,"15min":3.2629999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":57.313000000000002,"5min":57.313000000000002,"15min":57.313000000000002},"last":65.575000000000003},{"interface":"front","average":{"1min":11.608000000000001,"5min":7.4400000000000004,"15min":2.9350000000000001},"min":{"1min":1.5469999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":53.343000000000004,"5min":68.238,"15min":68.238},"last":31.77}]},{"osd":5,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.965,"5min":6.7839999999999998,"15min":2.722},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":53.381,"5min":53.381,"15min":53.381},"last":31.806000000000001},{"interface":"front","average":{"1min":6.71,"5min":7.867,"15min":3.1309999999999998},"min":{"1min":0.47099999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":19.084,"5min":36.213999999999999,"15min":36.213999999999999},"last":10.901}]},{"osd":6,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.84,"5min":7.8650000000000002,"15min":3.0960000000000001},"min":{"1min":1.2270000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":57.411000000000001,"5min":57.411000000000001,"15min":57.411000000000001},"last":65.606999999999999},{"interface":"front","average":{"1min":6.2850000000000001,"5min":5.6790000000000003,"15min":2.3849999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":19.113,"5min":44.825000000000003,"15min":44.825000000000003},"last":5.8620000000000001}]},{"osd":7,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":9.2200000000000006,"15min":3.5830000000000002},"min":{"1min":0.59399999999999997,"5min":0.375,"15min":0.375},"max":{"1min":63.896999999999998,"5min":63.896999999999998,"15min":63.896999999999998},"last":65.638999999999996},{"interface":"front","average":{"1min":9.6880000000000006,"5min":7.1580000000000004,"15min":2.9129999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":53.182000000000002,"5min":53.182000000000002,"15min":53.182000000000002},"last":2.2759999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9930000000000003,"5min":7.0389999999999997,"15min":2.7829999999999999},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":23.896000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":6.0339999999999998,"5min":7.4790000000000001,"15min":2.956},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":23.943000000000001,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5129999999999999,"5min":6.2370000000000001,"15min":2.8029999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.300999999999998,"5min":40.901000000000003,"15min":40.901000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":5.2530000000000001,"5min":5.7229999999999999,"15min":2.5539999999999998},"min":{"1min":0.497,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":17.434999999999999,"5min":33.162999999999997,"15min":39.137999999999998},"last":0.76600000000000001}]},{"osd":2,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.7759999999999998,"5min":6.0620000000000003,"15min":2.7469999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":18.202000000000002,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":5.8650000000000002,"5min":5.1820000000000004,"15min":2.4100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":19.893999999999998,"5min":25.616,"15min":58.899000000000001},"last":0.747}]},{"osd":3,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":6.9400000000000004,"5min":6.2759999999999998,"15min":2.6760000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":23.933,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":10.135,"5min":7.7439999999999998,"15min":3.2050000000000001},"min":{"1min":0.61299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":34.786000000000001,"5min":111.221,"15min":111.221},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5339999999999998,"5min":5.5279999999999996,"15min":2.2120000000000002},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":20.184000000000001,"5min":51.871000000000002,"15min":51.871000000000002},"last":0.442},{"interface":"front","average":{"1min":3.6890000000000001,"5min":4.633,"15min":1.9590000000000001},"min":{"1min":0.47899999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":23.858000000000001,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":4.5899999999999999,"5min":4.7770000000000001,"15min":1.9510000000000001},"min":{"1min":0.68999999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.905999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":5.673,"5min":5.524,"15min":2.2810000000000001},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.377},"max":{"1min":14.161,"5min":51.932000000000002,"15min":51.932000000000002},"last":0.495}]},{"osd":7,"last update":"Wed May 8 23:40:13 2024","interfaces":[{"interface":"back","average":{"1min":5.9699999999999998,"5min":4.2699999999999996,"15min":1.8340000000000001},"min":{"1min":0.47899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":19.039000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":7.4729999999999999,"5min":5.6150000000000002,"15min":2.339},"min":{"1min":0.55900000000000005,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":24.138000000000002,"5min":51.978000000000002,"15min":51.978000000000002},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.942,"5min":6.6719999999999997,"15min":2.6099999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":42.609000000000002,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":9.5069999999999997,"5min":5.5670000000000002,"15min":2.226},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.30199999999999999},"max":{"1min":44.439,"5min":44.439,"15min":44.439},"last":0.72399999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.8230000000000004,"5min":6.2409999999999997,"15min":2.8530000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32100000000000001},"max":{"1min":31.355,"5min":32.787999999999997,"15min":124.85599999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":7.867,"5min":6,"15min":2.4079999999999999},"min":{"1min":0.48499999999999999,"5min":0.317,"15min":0.317},"max":{"1min":31.385000000000002,"5min":41.060000000000002,"15min":41.060000000000002},"last":0.70299999999999996}]},{"osd":2,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":12.061999999999999,"5min":7.4779999999999998,"15min":2.8919999999999999},"min":{"1min":0.46700000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":44.247,"5min":44.247,"15min":44.247},"last":0.69099999999999995},{"interface":"front","average":{"1min":8.5579999999999998,"5min":5.3840000000000003,"15min":2.23},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.35499999999999998},"max":{"1min":36.612000000000002,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":19.018999999999998,"5min":7.3360000000000003,"15min":2.8769999999999998},"min":{"1min":0.49399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":79.974000000000004,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":12,"5min":10.103999999999999,"15min":3.8300000000000001},"min":{"1min":0.39900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":43.343000000000004,"5min":49.991999999999997,"15min":49.991999999999997},"last":0.59999999999999998}]},{"osd":4,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":11.17,"5min":5.2290000000000001,"15min":2.117},"min":{"1min":0.69299999999999995,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":43.305,"5min":43.305,"15min":43.305},"last":0.40699999999999997},{"interface":"front","average":{"1min":9.3439999999999994,"5min":3.9980000000000002,"15min":1.7529999999999999},"min":{"1min":0.38600000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":42.551000000000002,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":7.6980000000000004,"5min":7.0529999999999999,"15min":2.7629999999999999},"min":{"1min":0.53000000000000003,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":9.6489999999999991,"5min":5.4109999999999996,"15min":2.2120000000000002},"min":{"1min":0.60699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":46.505000000000003,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Wed May 8 23:40:16 2024","interfaces":[{"interface":"back","average":{"1min":8.0839999999999996,"5min":5.8360000000000003,"15min":2.4009999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":43.262,"5min":43.262,"15min":43.262},"last":0.48999999999999999},{"interface":"front","average":{"1min":9.3680000000000003,"5min":8.077,"15min":3.2120000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":44.018000000000001,"5min":80.302000000000007,"15min":80.302000000000007},"last":0.50900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4621972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:41:13.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:41:13.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.734+0000 7f0df539e700 1 Processor -- start 2024-05-08T23:41:13.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.734+0000 7f0df539e700 1 -- start start 2024-05-08T23:41:13.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df539e700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f0df00c92e0 0x7f0df00c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df539e700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0df0153f90 0x7f0df01543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df539e700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df0071300 con 0x7f0df00c96c0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df539e700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df0154900 con 0x7f0df0153f90 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df539e700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df015d770 con 0x7f0df00c92e0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.735+0000 7f0df4b9d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0df0153f90 0x7f0df01543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0def7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0df00c96c0 0x7f0df006b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40258/0 (socket says 172.21.15.33:40258) 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0def7fe700 1 -- 172.21.15.33:0/2785418516 learned_addr learned my addr 172.21.15.33:0/2785418516 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0deffff700 1 --2- 172.21.15.33:0/2785418516 >> v2:172.21.15.42:3300/0 conn(0x7f0df00c92e0 0x7f0df00c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0df4b9d700 1 -- 172.21.15.33:0/2785418516 >> v2:172.21.15.42:3300/0 conn(0x7f0df00c92e0 msgr2=0x7f0df00c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0df4b9d700 1 --2- 172.21.15.33:0/2785418516 >> v2:172.21.15.42:3300/0 conn(0x7f0df00c92e0 0x7f0df00c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0df4b9d700 1 -- 172.21.15.33:0/2785418516 >> v1:172.21.15.33:6789/0 conn(0x7f0df00c96c0 legacy=0x7f0df006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0deeffd700 1 -- 172.21.15.33:0/2785418516 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010898949 0 0) 0x7f0df0071300 con 0x7f0df00c96c0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0df4b9d700 1 -- 172.21.15.33:0/2785418516 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df015a180 con 0x7f0df0153f90 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0df4b9d700 1 --2- 172.21.15.33:0/2785418516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df0153f90 0x7f0df01543c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0df006fa90 tx=0x7f0de800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3ab81e5533922c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0deeffd700 1 -- 172.21.15.33:0/2785418516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de800e070 con 0x7f0df0153f90 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0deeffd700 1 -- 172.21.15.33:0/2785418516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de800bbd0 con 0x7f0df0153f90 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.736+0000 7f0deeffd700 1 -- 172.21.15.33:0/2785418516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8012430 con 0x7f0df0153f90 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 -- 172.21.15.33:0/2785418516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df0153f90 msgr2=0x7f0df01543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 --2- 172.21.15.33:0/2785418516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df0153f90 0x7f0df01543c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0df006fa90 tx=0x7f0de800b330 comp rx=0 tx=0).stop 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 -- 172.21.15.33:0/2785418516 shutdown_connections 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 --2- 172.21.15.33:0/2785418516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df0153f90 0x7f0df01543c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 --2- 172.21.15.33:0/2785418516 >> v2:172.21.15.42:3300/0 conn(0x7f0df00c92e0 0x7f0df00c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 -- 172.21.15.33:0/2785418516 >> 172.21.15.33:0/2785418516 conn(0x7f0df006ea90 msgr2=0x7f0df006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 -- 172.21.15.33:0/2785418516 shutdown_connections 2024-05-08T23:41:13.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.737+0000 7f0df539e700 1 -- 172.21.15.33:0/2785418516 wait complete. 2024-05-08T23:41:13.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.738+0000 7f0df539e700 1 Processor -- start 2024-05-08T23:41:13.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 -- start start 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df00c92e0 0x7f0df00c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 0x7f0df015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015d240 con 0x7f0df00c96c0 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df015cf60 con 0x7f0df00c92e0 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.739+0000 7f0df539e700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01f2a20 con 0x7f0df0153f90 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44362/0 (socket says 172.21.15.33:44362) 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 -- 172.21.15.33:0/446080373 learned_addr learned my addr 172.21.15.33:0/446080373 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0deffff700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df00c92e0 0x7f0df00c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0df4b9d700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 0x7f0df015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 -- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 msgr2=0x7f0df015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 0x7f0df015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 -- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df00c92e0 msgr2=0x7f0df00c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df00c92e0 0x7f0df00c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 -- 172.21.15.33:0/446080373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8009a70 con 0x7f0df00c96c0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0df4b9d700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 0x7f0df015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0def7fe700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c540 tx=0x7f0de400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4004c80 con 0x7f0df00c96c0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4004de0 con 0x7f0df00c96c0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df01f2d00 con 0x7f0df00c96c0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.740+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4007310 con 0x7f0df00c96c0 2024-05-08T23:41:13.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.741+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df01f32c0 con 0x7f0df00c96c0 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.742+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df006e480 con 0x7f0df00c96c0 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.742+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f0de4007470 con 0x7f0df00c96c0 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.742+0000 7f0decff9700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 0x7f0ddc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.743+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f0de4089060 con 0x7f0df00c96c0 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.743+0000 7f0deffff700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 0x7f0ddc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.743+0000 7f0deffff700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 0x7f0ddc06d180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006b20 tx=0x7f0de0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.747+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f0de4054170 con 0x7f0df00c96c0 2024-05-08T23:41:13.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.967+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0df0075350 con 0x7f0df00c96c0 2024-05-08T23:41:13.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.968+0000 7f0decff9700 1 -- 172.21.15.33:0/446080373 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16379 (secure 0 0 0) 0x7f0de4057790 con 0x7f0df00c96c0 2024-05-08T23:41:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:13.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:59.845491+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:41:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 msgr2=0x7f0ddc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 0x7f0ddc06d180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0de0006b20 tx=0x7f0de0008040 comp rx=0 tx=0).stop 2024-05-08T23:41:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 msgr2=0x7f0df0143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0de400c540 tx=0x7f0de400c850 comp rx=0 tx=0).stop 2024-05-08T23:41:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 shutdown_connections 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f0ddc06acc0 0x7f0ddc06d180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f0df0153f90 0x7f0df015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0df00c92e0 0x7f0df00c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 --2- 172.21.15.33:0/446080373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0df00c96c0 0x7f0df0143790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.970+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 >> 172.21.15.33:0/446080373 conn(0x7f0df006ea90 msgr2=0x7f0df0072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:13.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.971+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 shutdown_connections 2024-05-08T23:41:13.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:13.971+0000 7f0df539e700 1 -- 172.21.15.33:0/446080373 wait complete. 2024-05-08T23:41:14.982 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T23:41:14.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:15.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.139+0000 7f8ad2717700 1 Processor -- start 2024-05-08T23:41:15.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.139+0000 7f8ad2717700 1 -- start start 2024-05-08T23:41:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad2717700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f8acc15d370 0x7f8acc15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad2717700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 0x7f8acc15e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad2717700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8acc15b2a0 con 0x7f8acc15e5d0 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad2717700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8acc15dc00 con 0x7f8acc15d370 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad2717700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0cfdb0 con 0x7f8acc0caf50 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad0cb4700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8acc15e5d0 0x7f8acc0cb660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40264/0 (socket says 172.21.15.33:40264) 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8ad0cb4700 1 -- 172.21.15.33:0/1622894532 learned_addr learned my addr 172.21.15.33:0/1622894532 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8acbfff700 1 --2- 172.21.15.33:0/1622894532 >> v2:172.21.15.42:3300/0 conn(0x7f8acc15d370 0x7f8acc15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8acb7fe700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 0x7f8acc15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8acaffd700 1 -- 172.21.15.33:0/1622894532 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106922275 0 0) 0x7f8acc15b2a0 con 0x7f8acc15e5d0 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.140+0000 7f8acaffd700 1 -- 172.21.15.33:0/1622894532 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab80035b0 con 0x7f8acc15e5d0 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acbfff700 1 -- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 msgr2=0x7f8acc15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acbfff700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 0x7f8acc15e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acbfff700 1 -- 172.21.15.33:0/1622894532 >> v1:172.21.15.33:6789/0 conn(0x7f8acc15e5d0 legacy=0x7f8acc0cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acbfff700 1 -- 172.21.15.33:0/1622894532 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc0d0090 con 0x7f8acc15d370 2024-05-08T23:41:15.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acb7fe700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 0x7f8acc15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acbfff700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc15ad60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009840 tx=0x7f8abc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac8b3b051f95ac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acaffd700 1 -- 172.21.15.33:0/1622894532 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc011830 con 0x7f8acc15d370 2024-05-08T23:41:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acaffd700 1 -- 172.21.15.33:0/1622894532 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc011990 con 0x7f8acc15d370 2024-05-08T23:41:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.141+0000 7f8acaffd700 1 -- 172.21.15.33:0/1622894532 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc011d00 con 0x7f8acc15d370 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 -- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 msgr2=0x7f8acc15ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc15ad60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009840 tx=0x7f8abc009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 -- 172.21.15.33:0/1622894532 shutdown_connections 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc15ad60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 --2- 172.21.15.33:0/1622894532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc0caf50 0x7f8acc15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 -- 172.21.15.33:0/1622894532 >> 172.21.15.33:0/1622894532 conn(0x7f8acc0666c0 msgr2=0x7f8acc066ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 -- 172.21.15.33:0/1622894532 shutdown_connections 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.142+0000 7f8ad2717700 1 -- 172.21.15.33:0/1622894532 wait complete. 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.143+0000 7f8ad2717700 1 Processor -- start 2024-05-08T23:41:15.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.143+0000 7f8ad2717700 1 -- start start 2024-05-08T23:41:15.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 0x7f8acc159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc15e5d0 0x7f8acc157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc157cf0 con 0x7f8acc0caf50 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc1f37f0 con 0x7f8acc15d370 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad2717700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc157ee0 con 0x7f8acc15e5d0 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8acbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 0x7f8acc159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8acbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 0x7f8acc159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44378/0 (socket says 172.21.15.33:44378) 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8acbfff700 1 -- 172.21.15.33:0/872603852 learned_addr learned my addr 172.21.15.33:0/872603852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8acb7fe700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.144+0000 7f8ad0cb4700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc15e5d0 0x7f8acc157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 -- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc15e5d0 msgr2=0x7f8acc157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f8acc15e5d0 0x7f8acc157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 -- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 msgr2=0x7f8acc156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8acc15d370 0x7f8acc156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40094a0 con 0x7f8acc0caf50 2024-05-08T23:41:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8acbfff700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 0x7f8acc159740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009b90 tx=0x7f8abc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc004550 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc01f070 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0094a0 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0038e0 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.145+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc1f2ec0 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.146+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8acc15d890 con 0x7f8acc0caf50 2024-05-08T23:41:15.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.146+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f8abc003a40 con 0x7f8acc0caf50 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ac8ff9700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8aac06ac90 0x7f8aac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f8abc021070 con 0x7f8acc0caf50 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8acb7fe700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8aac06ac90 0x7f8aac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ac8ff9700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f8aac073110 0x7f8aac0755b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aac075c80 con 0x7f8aac073110 2024-05-08T23:41:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8abc08f2c0 con 0x7f8acc0caf50 2024-05-08T23:41:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8ad0cb4700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f8aac073110 0x7f8aac0755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.147+0000 7f8acb7fe700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8aac06ac90 0x7f8aac06d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8acc1583d0 tx=0x7f8ab4009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.148+0000 7f8ad0cb4700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f8aac073110 0x7f8aac0755b0 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.148+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== osd.0 v2:172.21.15.33:6800/47117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8aac075c80 con 0x7f8aac073110 2024-05-08T23:41:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.164+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 --> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8acc0d0650 con 0x7f8aac073110 2024-05-08T23:41:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ac8ff9700 1 -- 172.21.15.33:0/872603852 <== osd.0 v2:172.21.15.33:6800/47117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8ac4001f00 con 0x7f8aac073110 2024-05-08T23:41:15.161 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:15.161 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f8aac073110 msgr2=0x7f8aac0755b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6800/47117,v1:172.21.15.33:6801/47117] conn(0x7f8aac073110 0x7f8aac0755b0 crc :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8aac06ac90 msgr2=0x7f8aac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f8aac06ac90 0x7f8aac06d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8acc1583d0 tx=0x7f8ab4009210 comp rx=0 tx=0).stop 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 msgr2=0x7f8acc159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 --2- 172.21.15.33:0/872603852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8acc0caf50 0x7f8acc159740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009b90 tx=0x7f8abc004750 comp rx=0 tx=0).stop 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad0cb4700 1 -- 172.21.15.33:0/872603852 reap_dead start 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 shutdown_connections 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 >> 172.21.15.33:0/872603852 conn(0x7f8acc0666c0 msgr2=0x7f8acc15b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 shutdown_connections 2024-05-08T23:41:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.165+0000 7f8ad2717700 1 -- 172.21.15.33:0/872603852 wait complete. 2024-05-08T23:41:15.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T23:41:15.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.317+0000 7f1e3d0f1700 1 Processor -- start 2024-05-08T23:41:15.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.317+0000 7f1e3d0f1700 1 -- start start 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3d0f1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3d0f1700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1e380c85a0 0x7f1e380c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3d0f1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e38070450 con 0x7f1e380c38a0 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3d0f1700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380c2e90 con 0x7f1e380c85a0 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3d0f1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e38070730 con 0x7f1e38152350 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3659c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1e380c38a0 0x7f1e380c9520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40266/0 (socket says 172.21.15.33:40266) 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3659c700 1 -- 172.21.15.33:0/3575859050 learned_addr learned my addr 172.21.15.33:0/3575859050 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3759e700 1 --2- 172.21.15.33:0/3575859050 >> v2:172.21.15.42:3300/0 conn(0x7f1e380c85a0 0x7f1e380c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e36d9d700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e35d9b700 1 -- 172.21.15.33:0/3575859050 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535865687 0 0) 0x7f1e38070450 con 0x7f1e380c38a0 2024-05-08T23:41:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e35d9b700 1 -- 172.21.15.33:0/3575859050 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e240035b0 con 0x7f1e380c38a0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3759e700 1 -- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 msgr2=0x7f1e38158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3759e700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3759e700 1 -- 172.21.15.33:0/3575859050 >> v1:172.21.15.33:6789/0 conn(0x7f1e380c38a0 legacy=0x7f1e380c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.318+0000 7f1e3759e700 1 -- 172.21.15.33:0/3575859050 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38070a10 con 0x7f1e380c85a0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3759e700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c8a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009d90 tx=0x7f1e3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f75c4613b89bf24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e35d9b700 1 -- 172.21.15.33:0/3575859050 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000e070 con 0x7f1e380c85a0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e35d9b700 1 -- 172.21.15.33:0/3575859050 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000bb20 con 0x7f1e380c85a0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e35d9b700 1 -- 172.21.15.33:0/3575859050 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30012430 con 0x7f1e380c85a0 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 msgr2=0x7f1e380c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c8a10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009d90 tx=0x7f1e3000b330 comp rx=0 tx=0).stop 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/3575859050 shutdown_connections 2024-05-08T23:41:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c8a10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/3575859050 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.319+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/3575859050 >> 172.21.15.33:0/3575859050 conn(0x7f1e380666c0 msgr2=0x7f1e38066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.320+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/3575859050 shutdown_connections 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.320+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/3575859050 wait complete. 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.321+0000 7f1e3d0f1700 1 Processor -- start 2024-05-08T23:41:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.321+0000 7f1e3d0f1700 1 -- start start 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381f09b0 con 0x7f1e380c38a0 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c6850 con 0x7f1e380c85a0 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3d0f1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380ce1b0 con 0x7f1e38152350 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e36d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44388/0 (socket says 172.21.15.33:44388) 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e36d9d700 1 -- 172.21.15.33:0/2716986859 learned_addr learned my addr 172.21.15.33:0/2716986859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3659c700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.322+0000 7f1e3759e700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 -- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 msgr2=0x7f1e38146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 -- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 msgr2=0x7f1e380c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 -- 172.21.15.33:0/2716986859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009a70 con 0x7f1e380c38a0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e36d9d700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e870 tx=0x7f1e2000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20009590 con 0x7f1e380c38a0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20013680 con 0x7f1e380c38a0 2024-05-08T23:41:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200139f0 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e381efe30 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.323+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e381f03f0 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.324+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38075460 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.325+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1e20013b50 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.325+0000 7f1e2f7fe700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.325+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1e200892e0 con 0x7f1e380c38a0 2024-05-08T23:41:15.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.325+0000 7f1e3659c700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.326+0000 7f1e3659c700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 0x7f1e1806d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e381471f0 tx=0x7f1e28007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.330+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1e20054360 con 0x7f1e380c38a0 2024-05-08T23:41:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.548+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e38070110 con 0x7f1e1806ac90 2024-05-08T23:41:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.548+0000 7f1e2f7fe700 1 -- 172.21.15.33:0/2716986859 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e280058e0 con 0x7f1e1806ac90 2024-05-08T23:41:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 msgr2=0x7f1e1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 0x7f1e1806d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1e381471f0 tx=0x7f1e28007dc0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 msgr2=0x7f1e3806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e870 tx=0x7f1e2000eb80 comp rx=0 tx=0).stop 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 shutdown_connections 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1e1806ac90 0x7f1e1806d150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1e38152350 0x7f1e38146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e380c38a0 0x7f1e3806d1e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 --2- 172.21.15.33:0/2716986859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e380c85a0 0x7f1e380c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 >> 172.21.15.33:0/2716986859 conn(0x7f1e380666c0 msgr2=0x7f1e380cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 shutdown_connections 2024-05-08T23:41:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.550+0000 7f1e3d0f1700 1 -- 172.21.15.33:0/2716986859 wait complete. 2024-05-08T23:41:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T23:41:15.561 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T23:41:15.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.707+0000 7fd795928700 1 Processor -- start 2024-05-08T23:41:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.707+0000 7fd795928700 1 -- start start 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd795928700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd790154610 0x7fd7900cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd795928700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900c5ed0 con 0x7fd7900cffd0 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd795928700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79015ac80 con 0x7fd7900c6360 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd795928700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79015af60 con 0x7fd790154610 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd78effd700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd790154610 0x7fd7900cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd78f7fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd7900cffd0 0x7fd790164d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33352/0 (socket says 172.21.15.33:33352) 2024-05-08T23:41:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.708+0000 7fd78f7fe700 1 -- 172.21.15.33:0/484283171 learned_addr learned my addr 172.21.15.33:0/484283171 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78effd700 1 -- 172.21.15.33:0/484283171 >> v1:172.21.15.42:6789/0 conn(0x7fd7900c6360 legacy=0x7fd79015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78effd700 1 -- 172.21.15.33:0/484283171 >> v1:172.21.15.33:6789/0 conn(0x7fd7900cffd0 legacy=0x7fd790164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78effd700 1 -- 172.21.15.33:0/484283171 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79015b240 con 0x7fd790154610 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78dffb700 1 -- 172.21.15.33:0/484283171 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887317423 0 0) 0x7fd7900c5ed0 con 0x7fd7900cffd0 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78effd700 1 --2- 172.21.15.33:0/484283171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790154610 0x7fd7900cab60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd780009840 tx=0x7fd780009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829d75c454c9e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78dffb700 1 -- 172.21.15.33:0/484283171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780011830 con 0x7fd790154610 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78dffb700 1 -- 172.21.15.33:0/484283171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780011990 con 0x7fd790154610 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.709+0000 7fd78dffb700 1 -- 172.21.15.33:0/484283171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780011d20 con 0x7fd790154610 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 -- 172.21.15.33:0/484283171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790154610 msgr2=0x7fd7900cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 --2- 172.21.15.33:0/484283171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790154610 0x7fd7900cab60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd780009840 tx=0x7fd780009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 -- 172.21.15.33:0/484283171 shutdown_connections 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 --2- 172.21.15.33:0/484283171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790154610 0x7fd7900cab60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 -- 172.21.15.33:0/484283171 >> 172.21.15.33:0/484283171 conn(0x7fd79006cb90 msgr2=0x7fd79006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 -- 172.21.15.33:0/484283171 shutdown_connections 2024-05-08T23:41:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.710+0000 7fd795928700 1 -- 172.21.15.33:0/484283171 wait complete. 2024-05-08T23:41:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.711+0000 7fd795928700 1 Processor -- start 2024-05-08T23:41:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 -- start start 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7900c6360 0x7fd7900cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 0x7fd7900b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd790154610 0x7fd7900b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901f0ea0 con 0x7fd7900cffd0 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900c7580 con 0x7fd790154610 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.712+0000 7fd795928700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901f0290 con 0x7fd7900c6360 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 0x7fd7900b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 0x7fd7900b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60694/0 (socket says 172.21.15.33:60694) 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 -- 172.21.15.33:0/962895593 learned_addr learned my addr 172.21.15.33:0/962895593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78f7fe700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd790154610 0x7fd7900b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78effd700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7900c6360 0x7fd7900cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 -- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7900c6360 msgr2=0x7fd7900cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd7900c6360 0x7fd7900cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 -- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd790154610 msgr2=0x7fd7900b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd790154610 0x7fd7900b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800094a0 con 0x7fd7900cffd0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd78e7fc700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 0x7fd7900b3e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c4e0 tx=0x7fd78400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840052d0 con 0x7fd7900cffd0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784005430 con 0x7fd7900cffd0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7901f0570 con 0x7fd7900cffd0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840038c0 con 0x7fd7900cffd0 2024-05-08T23:41:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.713+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7901f0b30 con 0x7fd7900cffd0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.714+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd79015aa40 con 0x7fd7900cffd0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.715+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd784003a20 con 0x7fd7900cffd0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.715+0000 7fd777fff700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd77c06acc0 0x7fd77c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.715+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd7840887d0 con 0x7fd7900cffd0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.715+0000 7fd78effd700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd77c06acc0 0x7fd77c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.715+0000 7fd777fff700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fd77c073140 0x7fd77c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.716+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd77c075cb0 con 0x7fd77c073140 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.716+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd78408d2c0 con 0x7fd7900cffd0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.716+0000 7fd78f7fe700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fd77c073140 0x7fd77c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.716+0000 7fd78effd700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd77c06acc0 0x7fd77c06d180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd7800090b0 tx=0x7fd78001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.716+0000 7fd78f7fe700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fd77c073140 0x7fd77c0755e0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.717+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== osd.1 v2:172.21.15.33:6816/47120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd77c075cb0 con 0x7fd77c073140 2024-05-08T23:41:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 --> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd7900cb0a0 con 0x7fd77c073140 2024-05-08T23:41:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd777fff700 1 -- 172.21.15.33:0/962895593 <== osd.1 v2:172.21.15.33:6816/47120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd788007260 con 0x7fd77c073140 2024-05-08T23:41:15.730 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:15.730 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:15.730 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fd77c073140 msgr2=0x7fd77c0755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6816/47120,v1:172.21.15.33:6817/47120] conn(0x7fd77c073140 0x7fd77c0755e0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd77c06acc0 msgr2=0x7fd77c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd77c06acc0 0x7fd77c06d180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd7800090b0 tx=0x7fd78001c040 comp rx=0 tx=0).stop 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 msgr2=0x7fd7900b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 --2- 172.21.15.33:0/962895593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900cffd0 0x7fd7900b3e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c4e0 tx=0x7fd78400c7f0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd78f7fe700 1 -- 172.21.15.33:0/962895593 reap_dead start 2024-05-08T23:41:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 shutdown_connections 2024-05-08T23:41:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 >> 172.21.15.33:0/962895593 conn(0x7fd79006cb90 msgr2=0x7fd7900ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 shutdown_connections 2024-05-08T23:41:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.733+0000 7fd795928700 1 -- 172.21.15.33:0/962895593 wait complete. 2024-05-08T23:41:15.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T23:41:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.885+0000 7f1befcc7700 1 Processor -- start 2024-05-08T23:41:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.885+0000 7f1befcc7700 1 -- start start 2024-05-08T23:41:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f1be8154130 0x7f1be815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1be8154510 0x7f1be80667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be80c6ac0 0x7f1be80c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be815e890 con 0x7f1be8154510 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be815b260 con 0x7f1be8154130 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1befcc7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8066d20 con 0x7f1be80c6ac0 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1bed262700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1be8154510 0x7f1be80667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1bed262700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1be8154510 0x7f1be80667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60696/0 (socket says 172.21.15.33:60696) 2024-05-08T23:41:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.886+0000 7f1bed262700 1 -- 172.21.15.33:0/3283712926 learned_addr learned my addr 172.21.15.33:0/3283712926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1beda63700 1 --2- 172.21.15.33:0/3283712926 >> v2:172.21.15.42:3300/0 conn(0x7f1be8154130 0x7f1be815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bee264700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be80c6ac0 0x7f1be80c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 -- 172.21.15.33:0/3283712926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be80c6ac0 msgr2=0x7f1be80c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be80c6ac0 0x7f1be80c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 -- 172.21.15.33:0/3283712926 >> v2:172.21.15.42:3300/0 conn(0x7f1be8154130 msgr2=0x7f1be815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 --2- 172.21.15.33:0/3283712926 >> v2:172.21.15.42:3300/0 conn(0x7f1be8154130 0x7f1be815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 -- 172.21.15.33:0/3283712926 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be815da30 con 0x7f1be8154510 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1bed262700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be8154510 0x7f1be80667e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1be806f380 tx=0x7f1be4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85388c8472791d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1beca61700 1 -- 172.21.15.33:0/3283712926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4011830 con 0x7f1be8154510 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1beca61700 1 -- 172.21.15.33:0/3283712926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be4011990 con 0x7f1be8154510 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.887+0000 7f1beca61700 1 -- 172.21.15.33:0/3283712926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4011d20 con 0x7f1be8154510 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 -- 172.21.15.33:0/3283712926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be8154510 msgr2=0x7f1be80667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be8154510 0x7f1be80667e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1be806f380 tx=0x7f1be4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 -- 172.21.15.33:0/3283712926 shutdown_connections 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be80c6ac0 0x7f1be80c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 --2- 172.21.15.33:0/3283712926 >> v2:172.21.15.42:3300/0 conn(0x7f1be8154130 0x7f1be815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 --2- 172.21.15.33:0/3283712926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be8154510 0x7f1be80667e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 -- 172.21.15.33:0/3283712926 >> 172.21.15.33:0/3283712926 conn(0x7f1be8067370 msgr2=0x7f1be806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 -- 172.21.15.33:0/3283712926 shutdown_connections 2024-05-08T23:41:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.888+0000 7f1befcc7700 1 -- 172.21.15.33:0/3283712926 wait complete. 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.889+0000 7f1befcc7700 1 Processor -- start 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.889+0000 7f1befcc7700 1 -- start start 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be8154130 0x7f1be81efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1be8154510 0x7f1be81f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c3fd0 con 0x7f1be80c6ac0 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80c4250 con 0x7f1be8154510 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1befcc7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8159f40 con 0x7f1be8154130 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1beda63700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1beda63700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60712/0 (socket says 172.21.15.33:60712) 2024-05-08T23:41:15.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1beda63700 1 -- 172.21.15.33:0/240830572 learned_addr learned my addr 172.21.15.33:0/240830572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1bed262700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be8154130 0x7f1be81efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.890+0000 7f1bee264700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1be8154510 0x7f1be81f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 -- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be8154130 msgr2=0x7f1be81efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be8154130 0x7f1be81efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 -- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1be8154510 msgr2=0x7f1be81f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1be8154510 0x7f1be81f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 -- 172.21.15.33:0/240830572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009550 con 0x7f1be80c6ac0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1beda63700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e870 tx=0x7f1bdc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc009590 con 0x7f1be80c6ac0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc013680 con 0x7f1be80c6ac0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be8153640 con 0x7f1be80c6ac0 2024-05-08T23:41:15.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0139f0 con 0x7f1be80c6ac0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.891+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be81538e0 con 0x7f1be80c6ac0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.892+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be8075460 con 0x7f1be80c6ac0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.893+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1bdc013b50 con 0x7f1be80c6ac0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.893+0000 7f1bda7fc700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 0x7f1bcc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.893+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1bdc089190 con 0x7f1be80c6ac0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.893+0000 7f1bed262700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 0x7f1bcc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:15.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.894+0000 7f1bed262700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 0x7f1bcc06d180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009c10 tx=0x7f1be4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:15.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:15.898+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1bdc054330 con 0x7f1be80c6ac0 2024-05-08T23:41:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.117+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1be81ef920 con 0x7f1bcc06acc0 2024-05-08T23:41:16.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.118+0000 7f1bda7fc700 1 -- 172.21.15.33:0/240830572 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1be81ef920 con 0x7f1bcc06acc0 2024-05-08T23:41:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 msgr2=0x7f1bcc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 0x7f1bcc06d180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009c10 tx=0x7f1be4008030 comp rx=0 tx=0).stop 2024-05-08T23:41:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 msgr2=0x7f1be815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e870 tx=0x7f1bdc00eb80 comp rx=0 tx=0).stop 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 shutdown_connections 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1bcc06acc0 0x7f1bcc06d180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1be8154510 0x7f1be81f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be80c6ac0 0x7f1be815c930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 --2- 172.21.15.33:0/240830572 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1be8154130 0x7f1be81efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 >> 172.21.15.33:0/240830572 conn(0x7f1be8067370 msgr2=0x7f1be8151ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 shutdown_connections 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.120+0000 7f1befcc7700 1 -- 172.21.15.33:0/240830572 wait complete. 2024-05-08T23:41:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T23:41:16.131 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T23:41:16.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.275+0000 7fc69d89c700 1 Processor -- start 2024-05-08T23:41:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.275+0000 7fc69d89c700 1 -- start start 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc69d89c700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fc698070800 0x7fc69815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc69d89c700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69815a1b0 con 0x7fc6980c8e70 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc69d89c700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc698071a90 con 0x7fc6980c9250 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc69d89c700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69815b170 con 0x7fc698070800 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc696ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc6980c8e70 0x7fc6980666c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33364/0 (socket says 172.21.15.33:33364) 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.276+0000 7fc696ffd700 1 -- 172.21.15.33:0/323383645 learned_addr learned my addr 172.21.15.33:0/323383645 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc6967fc700 1 --2- 172.21.15.33:0/323383645 >> v2:172.21.15.42:3300/0 conn(0x7fc698070800 0x7fc69815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667597235 0 0) 0x7fc69815a1b0 con 0x7fc6980c8e70 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6840035b0 con 0x7fc6980c8e70 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc6967fc700 1 -- 172.21.15.33:0/323383645 >> v1:172.21.15.42:6789/0 conn(0x7fc6980c9250 legacy=0x7fc698153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354538756 0 0) 0x7fc6840035b0 con 0x7fc6980c8e70 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc6967fc700 1 -- 172.21.15.33:0/323383645 >> v1:172.21.15.33:6789/0 conn(0x7fc6980c8e70 legacy=0x7fc6980666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc6967fc700 1 -- 172.21.15.33:0/323383645 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698071d70 con 0x7fc698070800 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc6967fc700 1 --2- 172.21.15.33:0/323383645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc698070800 0x7fc69815ac30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6980c80a0 tx=0x7fc680009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b7add8431082b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.277+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688001670 con 0x7fc698070800 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680004030 con 0x7fc698070800 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc695ffb700 1 -- 172.21.15.33:0/323383645 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6800043a0 con 0x7fc698070800 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 -- 172.21.15.33:0/323383645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc698070800 msgr2=0x7fc69815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 --2- 172.21.15.33:0/323383645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc698070800 0x7fc69815ac30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6980c80a0 tx=0x7fc680009b20 comp rx=0 tx=0).stop 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 -- 172.21.15.33:0/323383645 shutdown_connections 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 --2- 172.21.15.33:0/323383645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc698070800 0x7fc69815ac30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 -- 172.21.15.33:0/323383645 >> 172.21.15.33:0/323383645 conn(0x7fc69806a990 msgr2=0x7fc69806ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 -- 172.21.15.33:0/323383645 shutdown_connections 2024-05-08T23:41:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.278+0000 7fc69d89c700 1 -- 172.21.15.33:0/323383645 wait complete. 2024-05-08T23:41:16.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.279+0000 7fc69d89c700 1 Processor -- start 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 -- start start 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc698070800 0x7fc698156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6980c8e70 0x7fc69814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 0x7fc698154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698154b40 con 0x7fc6980c9250 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698156cf0 con 0x7fc6980c8e70 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.280+0000 7fc69d89c700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc698156730 con 0x7fc698070800 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 0x7fc698154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 0x7fc698154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60714/0 (socket says 172.21.15.33:60714) 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 -- 172.21.15.33:0/3113765528 learned_addr learned my addr 172.21.15.33:0/3113765528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6967fc700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6980c8e70 0x7fc69814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc696ffd700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc698070800 0x7fc698156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 -- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc698070800 msgr2=0x7fc698156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fc698070800 0x7fc698156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 -- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6980c8e70 msgr2=0x7fc69814bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6980c8e70 0x7fc69814bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800094a0 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc6977fe700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 0x7fc698154100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c7f0 tx=0x7fc69000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690003e30 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc690005070 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690005420 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc698156a10 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.281+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6981f06c0 con 0x7fc6980c9250 2024-05-08T23:41:16.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.282+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6981514f0 con 0x7fc6980c9250 2024-05-08T23:41:16.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.283+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fc690005580 con 0x7fc6980c9250 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.283+0000 7fc68ffff700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc67806ac90 0x7fc67806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.283+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fc6900891e0 con 0x7fc6980c9250 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.283+0000 7fc696ffd700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc67806ac90 0x7fc67806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc68ffff700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fc678073110 0x7fc6780755b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc678075c80 con 0x7fc678073110 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69008b8a0 con 0x7fc6980c9250 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc6967fc700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fc678073110 0x7fc6780755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc696ffd700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc67806ac90 0x7fc67806d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc688001eb0 tx=0x7fc688001c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.284+0000 7fc6967fc700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fc678073110 0x7fc6780755b0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.285+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== osd.2 v2:172.21.15.33:6808/47118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc678075c80 con 0x7fc678073110 2024-05-08T23:41:16.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 --> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc698152e40 con 0x7fc678073110 2024-05-08T23:41:16.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc68ffff700 1 -- 172.21.15.33:0/3113765528 <== osd.2 v2:172.21.15.33:6808/47118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc698152e40 con 0x7fc678073110 2024-05-08T23:41:16.297 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:16.297 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:16.298 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:16.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fc678073110 msgr2=0x7fc6780755b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6808/47118,v1:172.21.15.33:6809/47118] conn(0x7fc678073110 0x7fc6780755b0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc67806ac90 msgr2=0x7fc67806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fc67806ac90 0x7fc67806d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc688001eb0 tx=0x7fc688001c80 comp rx=0 tx=0).stop 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 msgr2=0x7fc698154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 --2- 172.21.15.33:0/3113765528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6980c9250 0x7fc698154100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c7f0 tx=0x7fc69000cbb0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc6977fe700 1 -- 172.21.15.33:0/3113765528 reap_dead start 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 shutdown_connections 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.301+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 >> 172.21.15.33:0/3113765528 conn(0x7fc69806a990 msgr2=0x7fc698151fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.302+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 shutdown_connections 2024-05-08T23:41:16.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.302+0000 7fc69d89c700 1 -- 172.21.15.33:0/3113765528 wait complete. 2024-05-08T23:41:16.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T23:41:16.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.454+0000 7ff19cf41700 1 Processor -- start 2024-05-08T23:41:16.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.454+0000 7ff19cf41700 1 -- start start 2024-05-08T23:41:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.455+0000 7ff19cf41700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1980cdd70 0x7ff1980c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.455+0000 7ff19cf41700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1980ce150 0x7ff1980c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.455+0000 7ff19cf41700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19815e6e0 con 0x7ff198153fd0 2024-05-08T23:41:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.455+0000 7ff19cf41700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980c9090 con 0x7ff1980cdd70 2024-05-08T23:41:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19cf41700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980c3b60 con 0x7ff1980ce150 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1980cdd70 0x7ff1980c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff196d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff198153fd0 0x7ff198154440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33370/0 (socket says 172.21.15.33:33370) 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff196d9d700 1 -- 172.21.15.33:0/2618474881 learned_addr learned my addr 172.21.15.33:0/2618474881 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff195d9b700 1 --2- 172.21.15.33:0/2618474881 >> v2:172.21.15.42:3300/0 conn(0x7ff1980ce150 0x7ff1980c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 -- 172.21.15.33:0/2618474881 >> v2:172.21.15.42:3300/0 conn(0x7ff1980ce150 msgr2=0x7ff1980c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 --2- 172.21.15.33:0/2618474881 >> v2:172.21.15.42:3300/0 conn(0x7ff1980ce150 0x7ff1980c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 -- 172.21.15.33:0/2618474881 >> v1:172.21.15.33:6789/0 conn(0x7ff198153fd0 legacy=0x7ff198154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 -- 172.21.15.33:0/2618474881 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19815db60 con 0x7ff1980cdd70 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19559a700 1 -- 172.21.15.33:0/2618474881 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587333661 0 0) 0x7ff19815e6e0 con 0x7ff198153fd0 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19659c700 1 --2- 172.21.15.33:0/2618474881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1980cdd70 0x7ff1980c9520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1980c4760 tx=0x7ff188009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634b70f04e9b2378 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19559a700 1 -- 172.21.15.33:0/2618474881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011830 con 0x7ff1980cdd70 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19559a700 1 -- 172.21.15.33:0/2618474881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188011990 con 0x7ff1980cdd70 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.456+0000 7ff19559a700 1 -- 172.21.15.33:0/2618474881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188011d20 con 0x7ff1980cdd70 2024-05-08T23:41:16.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 -- 172.21.15.33:0/2618474881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1980cdd70 msgr2=0x7ff1980c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 --2- 172.21.15.33:0/2618474881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1980cdd70 0x7ff1980c9520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1980c4760 tx=0x7ff188009b20 comp rx=0 tx=0).stop 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 -- 172.21.15.33:0/2618474881 shutdown_connections 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 --2- 172.21.15.33:0/2618474881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1980cdd70 0x7ff1980c9520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 --2- 172.21.15.33:0/2618474881 >> v2:172.21.15.42:3300/0 conn(0x7ff1980ce150 0x7ff1980c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 -- 172.21.15.33:0/2618474881 >> 172.21.15.33:0/2618474881 conn(0x7ff1980666c0 msgr2=0x7ff198066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 -- 172.21.15.33:0/2618474881 shutdown_connections 2024-05-08T23:41:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.457+0000 7ff19cf41700 1 -- 172.21.15.33:0/2618474881 wait complete. 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 Processor -- start 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 -- start start 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1980cdd70 0x7ff198159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1980ce150 0x7ff19806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19806c3b0 con 0x7ff198153fd0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff198159ac0 con 0x7ff1980cdd70 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.459+0000 7ff19cf41700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19806bb10 con 0x7ff1980ce150 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60732/0 (socket says 172.21.15.33:60732) 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 -- 172.21.15.33:0/1682610252 learned_addr learned my addr 172.21.15.33:0/1682610252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff19659c700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1980cdd70 0x7ff198159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff195d9b700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1980ce150 0x7ff19806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 -- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1980ce150 msgr2=0x7ff19806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1980ce150 0x7ff19806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 -- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1980cdd70 msgr2=0x7ff198159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1980cdd70 0x7ff198159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 -- 172.21.15.33:0/1682610252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880094a0 con 0x7ff198153fd0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff196d9d700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff19000cba0 tx=0x7ff19000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000aaa0 con 0x7ff198153fd0 2024-05-08T23:41:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19000ac20 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff198153490 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1900070d0 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.460+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19815a2c0 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.461+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1980c40c0 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.462+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff190007230 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.462+0000 7ff18effd700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 0x7ff17806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.462+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff190089110 con 0x7ff198153fd0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.463+0000 7ff19659c700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 0x7ff17806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.463+0000 7ff19659c700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 0x7ff17806d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff1880090b0 tx=0x7ff188008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.467+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff190054220 con 0x7ff198153fd0 2024-05-08T23:41:16.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.691+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff19815e400 con 0x7ff17806ac90 2024-05-08T23:41:16.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.691+0000 7ff18effd700 1 -- 172.21.15.33:0/1682610252 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff188011c00 con 0x7ff17806ac90 2024-05-08T23:41:16.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 msgr2=0x7ff17806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 0x7ff17806d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff1880090b0 tx=0x7ff188008030 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 msgr2=0x7ff1980ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff19000cba0 tx=0x7ff19000ceb0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 shutdown_connections 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff17806ac90 0x7ff17806d150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff198153fd0 0x7ff1980ca7e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1980cdd70 0x7ff198159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 --2- 172.21.15.33:0/1682610252 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1980ce150 0x7ff19806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 >> 172.21.15.33:0/1682610252 conn(0x7ff1980666c0 msgr2=0x7ff198066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 shutdown_connections 2024-05-08T23:41:16.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.694+0000 7ff19cf41700 1 -- 172.21.15.33:0/1682610252 wait complete. 2024-05-08T23:41:16.692 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T23:41:16.704 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T23:41:16.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:16.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.849+0000 7f9d6d3ad700 1 Processor -- start 2024-05-08T23:41:16.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.849+0000 7f9d6d3ad700 1 -- start start 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d6d3ad700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f9d6806ec30 0x7f9d68070120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d6d3ad700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9d68070660 0x7f9d680c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d6d3ad700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6806f250 con 0x7f9d681522f0 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d6d3ad700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6806e060 con 0x7f9d68070660 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d6d3ad700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6806e340 con 0x7f9d6806ec30 2024-05-08T23:41:16.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d677fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9d68070660 0x7f9d680c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d66ffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9d681522f0 0x7f9d68158b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33376/0 (socket says 172.21.15.33:33376) 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.850+0000 7f9d66ffd700 1 -- 172.21.15.33:0/317585536 learned_addr learned my addr 172.21.15.33:0/317585536 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d667fc700 1 --2- 172.21.15.33:0/317585536 >> v2:172.21.15.42:3300/0 conn(0x7f9d6806ec30 0x7f9d68070120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d677fe700 1 -- 172.21.15.33:0/317585536 >> v2:172.21.15.42:3300/0 conn(0x7f9d6806ec30 msgr2=0x7f9d68070120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d677fe700 1 --2- 172.21.15.33:0/317585536 >> v2:172.21.15.42:3300/0 conn(0x7f9d6806ec30 0x7f9d68070120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d677fe700 1 -- 172.21.15.33:0/317585536 >> v1:172.21.15.33:6789/0 conn(0x7f9d681522f0 legacy=0x7f9d68158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d677fe700 1 -- 172.21.15.33:0/317585536 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6806e620 con 0x7f9d68070660 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d677fe700 1 --2- 172.21.15.33:0/317585536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d680c45f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d680c39e0 tx=0x7f9d6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7225d260d998305f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d65ffb700 1 -- 172.21.15.33:0/317585536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000e070 con 0x7f9d68070660 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d65ffb700 1 -- 172.21.15.33:0/317585536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000bbd0 con 0x7f9d68070660 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.851+0000 7f9d65ffb700 1 -- 172.21.15.33:0/317585536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60012430 con 0x7f9d68070660 2024-05-08T23:41:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/317585536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 msgr2=0x7f9d680c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/317585536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d680c45f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d680c39e0 tx=0x7f9d6000b330 comp rx=0 tx=0).stop 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/317585536 shutdown_connections 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/317585536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d680c45f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/317585536 >> v2:172.21.15.42:3300/0 conn(0x7f9d6806ec30 0x7f9d68070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/317585536 >> 172.21.15.33:0/317585536 conn(0x7f9d68066cf0 msgr2=0x7f9d680721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/317585536 shutdown_connections 2024-05-08T23:41:16.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.852+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/317585536 wait complete. 2024-05-08T23:41:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.853+0000 7f9d6d3ad700 1 Processor -- start 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.853+0000 7f9d6d3ad700 1 -- start start 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9d6806ec30 0x7f9d681edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d68155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d681522f0 0x7f9d6815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68159850 con 0x7f9d68070660 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68159b30 con 0x7f9d681522f0 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d6d3ad700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68159e10 con 0x7f9d6806ec30 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d68155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d68155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60748/0 (socket says 172.21.15.33:60748) 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 -- 172.21.15.33:0/1498318893 learned_addr learned my addr 172.21.15.33:0/1498318893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d677fe700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d681522f0 0x7f9d6815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d66ffd700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9d6806ec30 0x7f9d681edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 -- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9d6806ec30 msgr2=0x7f9d681edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f9d6806ec30 0x7f9d681edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 -- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d681522f0 msgr2=0x7f9d6815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.854+0000 7f9d667fc700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d681522f0 0x7f9d6815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d667fc700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60009a70 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d667fc700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d68155190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c7f0 tx=0x7f9d5000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50011800 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50011960 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6815a0f0 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50011cd0 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.855+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d680cd7e0 con 0x7f9d68070660 2024-05-08T23:41:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.856+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d6814cb30 con 0x7f9d68070660 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.856+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f9d50004020 con 0x7f9d68070660 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d5f7fe700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9d4806ac90 0x7f9d4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f9d50088fe0 con 0x7f9d68070660 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d66ffd700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9d4806ac90 0x7f9d4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d5f7fe700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f9d48073110 0x7f9d480755b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d48075c80 con 0x7f9d48073110 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5008b5c0 con 0x7f9d68070660 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d677fe700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f9d48073110 0x7f9d480755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.857+0000 7f9d66ffd700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9d4806ac90 0x7f9d4806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d58000f60 tx=0x7f9d58007f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.858+0000 7f9d677fe700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f9d48073110 0x7f9d480755b0 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.858+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== osd.3 v2:172.21.15.33:6824/47140 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9d48075c80 con 0x7f9d48073110 2024-05-08T23:41:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 --> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d68159f80 con 0x7f9d48073110 2024-05-08T23:41:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d5f7fe700 1 -- 172.21.15.33:0/1498318893 <== osd.3 v2:172.21.15.33:6824/47140 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9d68159f80 con 0x7f9d48073110 2024-05-08T23:41:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:16.871 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:16.871 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f9d48073110 msgr2=0x7f9d480755b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6824/47140,v1:172.21.15.33:6825/47140] conn(0x7f9d48073110 0x7f9d480755b0 crc :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9d4806ac90 msgr2=0x7f9d4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f9d4806ac90 0x7f9d4806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9d58000f60 tx=0x7f9d58007f10 comp rx=0 tx=0).stop 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 msgr2=0x7f9d68155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 --2- 172.21.15.33:0/1498318893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d68070660 0x7f9d68155190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c7f0 tx=0x7f9d5000cb00 comp rx=0 tx=0).stop 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d677fe700 1 -- 172.21.15.33:0/1498318893 reap_dead start 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 shutdown_connections 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.874+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 >> 172.21.15.33:0/1498318893 conn(0x7f9d68066cf0 msgr2=0x7f9d68150170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.875+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 shutdown_connections 2024-05-08T23:41:16.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:16.875+0000 7f9d6d3ad700 1 -- 172.21.15.33:0/1498318893 wait complete. 2024-05-08T23:41:16.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T23:41:17.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.027+0000 7f1ce8c35700 1 Processor -- start 2024-05-08T23:41:17.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.027+0000 7f1ce8c35700 1 -- start start 2024-05-08T23:41:17.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce8c35700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce4154540 0x7f1ce40c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce8c35700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ce40712f0 0x7f1ce415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce8c35700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce40c2860 con 0x7f1ce406d980 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce8c35700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce40c3540 con 0x7f1ce40712f0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce8c35700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce40c9600 con 0x7f1ce4154540 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce1d9b700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ce40712f0 0x7f1ce415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce2d9d700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1ce406d980 0x7f1ce415de30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33388/0 (socket says 172.21.15.33:33388) 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.028+0000 7f1ce2d9d700 1 -- 172.21.15.33:0/3187499992 learned_addr learned my addr 172.21.15.33:0/3187499992 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce259c700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce4154540 0x7f1ce40c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce1d9b700 1 -- 172.21.15.33:0/3187499992 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce4154540 msgr2=0x7f1ce40c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce4154540 0x7f1ce40c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce1d9b700 1 -- 172.21.15.33:0/3187499992 >> v1:172.21.15.33:6789/0 conn(0x7f1ce406d980 legacy=0x7f1ce415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce1d9b700 1 -- 172.21.15.33:0/3187499992 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce40c8620 con 0x7f1ce40712f0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce40712f0 0x7f1ce415d8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009840 tx=0x7f1ccc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842cdefeab170430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce159a700 1 -- 172.21.15.33:0/3187499992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011830 con 0x7f1ce40712f0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce159a700 1 -- 172.21.15.33:0/3187499992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc011990 con 0x7f1ce40712f0 2024-05-08T23:41:17.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.029+0000 7f1ce159a700 1 -- 172.21.15.33:0/3187499992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011d20 con 0x7f1ce40712f0 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 -- 172.21.15.33:0/3187499992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce40712f0 msgr2=0x7f1ce415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce40712f0 0x7f1ce415d8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009840 tx=0x7f1ccc009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 -- 172.21.15.33:0/3187499992 shutdown_connections 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce4154540 0x7f1ce40c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/3187499992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce40712f0 0x7f1ce415d8f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 -- 172.21.15.33:0/3187499992 >> 172.21.15.33:0/3187499992 conn(0x7f1ce406ce60 msgr2=0x7f1ce406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 -- 172.21.15.33:0/3187499992 shutdown_connections 2024-05-08T23:41:17.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.030+0000 7f1ce8c35700 1 -- 172.21.15.33:0/3187499992 wait complete. 2024-05-08T23:41:17.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.032+0000 7f1ce8c35700 1 Processor -- start 2024-05-08T23:41:17.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.032+0000 7f1ce8c35700 1 -- start start 2024-05-08T23:41:17.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 0x7f1ce415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 0x7f1ce415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce406cc00 con 0x7f1ce406d980 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce406c710 con 0x7f1ce4154540 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce8c35700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce415d440 con 0x7f1ce40712f0 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60780/0 (socket says 172.21.15.33:60780) 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 -- 172.21.15.33:0/252857625 learned_addr learned my addr 172.21.15.33:0/252857625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce2d9d700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 0x7f1ce415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 0x7f1ce415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 -- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 msgr2=0x7f1ce415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 0x7f1ce415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 -- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 msgr2=0x7f1ce415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 0x7f1ce415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 -- 172.21.15.33:0/252857625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0094a0 con 0x7f1ce406d980 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 0x7f1ce415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce2d9d700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 0x7f1ce415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1ce259c700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400d800 tx=0x7f1cd400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.033+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40125f0 con 0x7f1ce406d980 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.034+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4012750 con 0x7f1ce406d980 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.034+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce406c920 con 0x7f1ce406d980 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.034+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4012b00 con 0x7f1ce406d980 2024-05-08T23:41:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.034+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce41f05a0 con 0x7f1ce406d980 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.035+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce4073770 con 0x7f1ce406d980 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.035+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1cd4012c60 con 0x7f1ce406d980 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.036+0000 7f1cdaffd700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 0x7f1cc406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.036+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 0x7f1cc406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.036+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1cd40898b0 con 0x7f1ce406d980 2024-05-08T23:41:17.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.036+0000 7f1ce1d9b700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 0x7f1cc406d180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ce415e7f0 tx=0x7f1ccc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.040+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1cd4054890 con 0x7f1ce406d980 2024-05-08T23:41:17.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.258+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ce40c8ba0 con 0x7f1cc406acc0 2024-05-08T23:41:17.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.259+0000 7f1cdaffd700 1 -- 172.21.15.33:0/252857625 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1ccc011b00 con 0x7f1cc406acc0 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 msgr2=0x7f1cc406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 0x7f1cc406d180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ce415e7f0 tx=0x7f1ccc008030 comp rx=0 tx=0).stop 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 msgr2=0x7f1ce406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400d800 tx=0x7f1cd400dbc0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 shutdown_connections 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1cc406acc0 0x7f1cc406d180 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1ce4154540 0x7f1ce415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ce406d980 0x7f1ce406bce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 --2- 172.21.15.33:0/252857625 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1ce40712f0 0x7f1ce415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 >> 172.21.15.33:0/252857625 conn(0x7f1ce406ce60 msgr2=0x7f1ce414fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 shutdown_connections 2024-05-08T23:41:17.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.261+0000 7f1ce8c35700 1 -- 172.21.15.33:0/252857625 wait complete. 2024-05-08T23:41:17.259 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T23:41:17.272 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T23:41:17.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:17.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.417+0000 7ff21dbe1700 1 Processor -- start 2024-05-08T23:41:17.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.417+0000 7ff21dbe1700 1 -- start start 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff21dbe1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 0x7ff21815a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff21dbe1700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 0x7ff2180702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff21dbe1700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21806fe50 con 0x7ff21806baa0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff21dbe1700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff218154a90 con 0x7ff21815d970 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff21dbe1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21815b320 con 0x7ff2181546b0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff217fff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff21806baa0 0x7ff21806bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33392/0 (socket says 172.21.15.33:33392) 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff217fff700 1 -- 172.21.15.33:0/2779108034 learned_addr learned my addr 172.21.15.33:0/2779108034 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff216ffd700 1 --2- 172.21.15.33:0/2779108034 >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 0x7ff2180702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.418+0000 7ff2177fe700 1 --2- 172.21.15.33:0/2779108034 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 0x7ff21815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157038685 0 0) 0x7ff21806fe50 con 0x7ff21806baa0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2040035b0 con 0x7ff21806baa0 2024-05-08T23:41:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2177fe700 1 -- 172.21.15.33:0/2779108034 >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 msgr2=0x7ff2180702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2177fe700 1 --2- 172.21.15.33:0/2779108034 >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 0x7ff2180702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2716027645 0 0) 0x7ff2040035b0 con 0x7ff21806baa0 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2177fe700 1 -- 172.21.15.33:0/2779108034 >> v1:172.21.15.33:6789/0 conn(0x7ff21806baa0 legacy=0x7ff21806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2177fe700 1 -- 172.21.15.33:0/2779108034 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21815aa80 con 0x7ff2181546b0 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff216ffd700 1 --2- 172.21.15.33:0/2779108034 >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 0x7ff2180702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2177fe700 1 --2- 172.21.15.33:0/2779108034 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 0x7ff21815a380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff208009840 tx=0x7ff208009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e54c6e0c65194d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2100019f0 con 0x7ff2181546b0 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff208004030 con 0x7ff2181546b0 2024-05-08T23:41:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.419+0000 7ff2167fc700 1 -- 172.21.15.33:0/2779108034 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080043a0 con 0x7ff2181546b0 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 -- 172.21.15.33:0/2779108034 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 msgr2=0x7ff21815a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/2779108034 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 0x7ff21815a380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff208009840 tx=0x7ff208009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 -- 172.21.15.33:0/2779108034 shutdown_connections 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/2779108034 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff2181546b0 0x7ff21815a380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/2779108034 >> v2:172.21.15.42:3300/0 conn(0x7ff21815d970 0x7ff2180702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 -- 172.21.15.33:0/2779108034 >> 172.21.15.33:0/2779108034 conn(0x7ff2180cde70 msgr2=0x7ff2180ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 -- 172.21.15.33:0/2779108034 shutdown_connections 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.420+0000 7ff21dbe1700 1 -- 172.21.15.33:0/2779108034 wait complete. 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.421+0000 7ff21dbe1700 1 Processor -- start 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 -- start start 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff21806baa0 0x7ff2180cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2181546b0 0x7ff2181f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 0x7ff2181f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c59d0 con 0x7ff21815d970 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c5410 con 0x7ff2181546b0 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff21dbe1700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180c56f0 con 0x7ff21806baa0 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff217fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 0x7ff2181f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff217fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 0x7ff2181f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60782/0 (socket says 172.21.15.33:60782) 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.422+0000 7ff217fff700 1 -- 172.21.15.33:0/287154682 learned_addr learned my addr 172.21.15.33:0/287154682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff216ffd700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2181546b0 0x7ff2181f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff2177fe700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff21806baa0 0x7ff2180cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 -- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff21806baa0 msgr2=0x7ff2180cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff21806baa0 0x7ff2180cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 -- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2181546b0 msgr2=0x7ff2181f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2181546b0 0x7ff2181f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000094a0 con 0x7ff21815d970 2024-05-08T23:41:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff217fff700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 0x7ff2181f0720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff210001ba0 tx=0x7ff21000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000aac0 con 0x7ff21815d970 2024-05-08T23:41:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000ac20 con 0x7ff21815d970 2024-05-08T23:41:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2080094a0 con 0x7ff21815d970 2024-05-08T23:41:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2100070d0 con 0x7ff21815d970 2024-05-08T23:41:17.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.423+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218153850 con 0x7ff21815d970 2024-05-08T23:41:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.424+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff21806fae0 con 0x7ff21815d970 2024-05-08T23:41:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.425+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff210007230 con 0x7ff21815d970 2024-05-08T23:41:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.425+0000 7ff20ffff700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1f806acc0 0x7ff1f806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.425+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff210089120 con 0x7ff21815d970 2024-05-08T23:41:17.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.425+0000 7ff2177fe700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1f806acc0 0x7ff1f806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.425+0000 7ff20ffff700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7ff1f8073140 0x7ff1f80755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.426+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1f8075cb0 con 0x7ff1f8073140 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.426+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff21008b700 con 0x7ff21815d970 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.426+0000 7ff216ffd700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7ff1f8073140 0x7ff1f80755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.426+0000 7ff2177fe700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1f806acc0 0x7ff1f806d180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2080090b0 tx=0x7ff208008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.426+0000 7ff216ffd700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7ff1f8073140 0x7ff1f80755e0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.427+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== osd.4 v2:172.21.15.42:6823/47110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff1f8075cb0 con 0x7ff1f8073140 2024-05-08T23:41:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.443+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 --> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2180c9300 con 0x7ff1f8073140 2024-05-08T23:41:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.443+0000 7ff20ffff700 1 -- 172.21.15.33:0/287154682 <== osd.4 v2:172.21.15.42:6823/47110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff200009d60 con 0x7ff1f8073140 2024-05-08T23:41:17.440 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:17.440 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:17.440 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:17.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7ff1f8073140 msgr2=0x7ff1f80755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.42:6823/47110,v1:172.21.15.42:6825/47110] conn(0x7ff1f8073140 0x7ff1f80755e0 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1f806acc0 msgr2=0x7ff1f806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1f806acc0 0x7ff1f806d180 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2080090b0 tx=0x7ff208008030 comp rx=0 tx=0).stop 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 msgr2=0x7ff2181f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 --2- 172.21.15.33:0/287154682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815d970 0x7ff2181f0720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff210001ba0 tx=0x7ff21000cf50 comp rx=0 tx=0).stop 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff217fff700 1 -- 172.21.15.33:0/287154682 reap_dead start 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 shutdown_connections 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 >> 172.21.15.33:0/287154682 conn(0x7ff2180cde70 msgr2=0x7ff2180ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 shutdown_connections 2024-05-08T23:41:17.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.444+0000 7ff21dbe1700 1 -- 172.21.15.33:0/287154682 wait complete. 2024-05-08T23:41:17.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T23:41:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.597+0000 7f303b506700 1 Processor -- start 2024-05-08T23:41:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.598+0000 7f303b506700 1 -- start start 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.598+0000 7f303b506700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3034151690 0x7f3034157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.598+0000 7f303b506700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 0x7f303406baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.598+0000 7f303b506700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3034151a70 con 0x7f30340c7f30 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.598+0000 7f303b506700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340c4480 con 0x7f3034151690 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f303b506700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3034071f10 con 0x7f30340c6d10 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3034151690 0x7f3034157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f3039aa3700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f30340c7f30 0x7f303415a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33400/0 (socket says 172.21.15.33:33400) 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f3039aa3700 1 -- 172.21.15.33:0/308390229 learned_addr learned my addr 172.21.15.33:0/308390229 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f3038aa1700 1 --2- 172.21.15.33:0/308390229 >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 0x7f303406baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f302bfff700 1 -- 172.21.15.33:0/308390229 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029946230 0 0) 0x7f302c001490 con 0x7f30340c7f30 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f302bfff700 1 -- 172.21.15.33:0/308390229 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30200035b0 con 0x7f30340c7f30 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 -- 172.21.15.33:0/308390229 >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 msgr2=0x7f303406baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 --2- 172.21.15.33:0/308390229 >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 0x7f303406baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 -- 172.21.15.33:0/308390229 >> v1:172.21.15.33:6789/0 conn(0x7f30340c7f30 legacy=0x7f303415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 -- 172.21.15.33:0/308390229 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340c76f0 con 0x7f3034151690 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f3038aa1700 1 --2- 172.21.15.33:0/308390229 >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 0x7f303406baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f30392a2700 1 --2- 172.21.15.33:0/308390229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034151690 0x7f3034157cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3034070990 tx=0x7f3030009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc61390cf44f974d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.599+0000 7f302bfff700 1 -- 172.21.15.33:0/308390229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030011830 con 0x7f3034151690 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f302bfff700 1 -- 172.21.15.33:0/308390229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030011990 con 0x7f3034151690 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f302bfff700 1 -- 172.21.15.33:0/308390229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030011d20 con 0x7f3034151690 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 -- 172.21.15.33:0/308390229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034151690 msgr2=0x7f3034157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 --2- 172.21.15.33:0/308390229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034151690 0x7f3034157cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3034070990 tx=0x7f3030009b20 comp rx=0 tx=0).stop 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 -- 172.21.15.33:0/308390229 shutdown_connections 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 --2- 172.21.15.33:0/308390229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3034151690 0x7f3034157cb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 --2- 172.21.15.33:0/308390229 >> v2:172.21.15.42:3300/0 conn(0x7f30340c6d10 0x7f303406baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 -- 172.21.15.33:0/308390229 >> 172.21.15.33:0/308390229 conn(0x7f303406d2b0 msgr2=0x7f303406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 -- 172.21.15.33:0/308390229 shutdown_connections 2024-05-08T23:41:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.600+0000 7f303b506700 1 -- 172.21.15.33:0/308390229 wait complete. 2024-05-08T23:41:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.601+0000 7f303b506700 1 Processor -- start 2024-05-08T23:41:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 -- start start 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 0x7f30340c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 0x7f30341565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341561c0 con 0x7f30340c6d10 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341efff0 con 0x7f3034151690 2024-05-08T23:41:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f303b506700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30341f02d0 con 0x7f30340c7f30 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f30392a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.602+0000 7f30392a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60804/0 (socket says 172.21.15.33:60804) 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 -- 172.21.15.33:0/3842475019 learned_addr learned my addr 172.21.15.33:0/3842475019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3039aa3700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 0x7f30341565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3038aa1700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 0x7f30340c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 -- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 msgr2=0x7f30340c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 0x7f30340c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 -- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 msgr2=0x7f30341565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 0x7f30341565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 -- 172.21.15.33:0/3842475019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30300094a0 con 0x7f30340c6d10 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3039aa3700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 0x7f30341565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3038aa1700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 0x7f30340c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f30392a2700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300036a0 con 0x7f30340c6d10 2024-05-08T23:41:17.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303001e070 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30341f05b0 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030003a60 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.603+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341f0b70 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.604+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034075460 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.605+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3030003bc0 con 0x7f30340c6d10 2024-05-08T23:41:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.605+0000 7f3029ffb700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 0x7f301806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.605+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f303008a4d0 con 0x7f30340c6d10 2024-05-08T23:41:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.605+0000 7f3038aa1700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 0x7f301806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.606+0000 7f3038aa1700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 0x7f301806d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f303414a040 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.610+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3030055540 con 0x7f30340c6d10 2024-05-08T23:41:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.835+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f30340718c0 con 0x7f301806ac90 2024-05-08T23:41:17.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.835+0000 7f3029ffb700 1 -- 172.21.15.33:0/3842475019 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3024009590 con 0x7f301806ac90 2024-05-08T23:41:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 msgr2=0x7f301806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 0x7f301806d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f303414a040 tx=0x7f3024008040 comp rx=0 tx=0).stop 2024-05-08T23:41:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 msgr2=0x7f303406c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030004750 comp rx=0 tx=0).stop 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 shutdown_connections 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f301806ac90 0x7f301806d150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3034151690 0x7f30341565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30340c6d10 0x7f303406c4d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 --2- 172.21.15.33:0/3842475019 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f30340c7f30 0x7f30340c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.837+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 >> 172.21.15.33:0/3842475019 conn(0x7f303406d2b0 msgr2=0x7f303414b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.838+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 shutdown_connections 2024-05-08T23:41:17.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.838+0000 7f303b506700 1 -- 172.21.15.33:0/3842475019 wait complete. 2024-05-08T23:41:17.836 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T23:41:17.848 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T23:41:17.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:17.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.995+0000 7ff15ec98700 1 Processor -- start 2024-05-08T23:41:17.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.995+0000 7ff15ec98700 1 -- start start 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ec98700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1580c8ae0 0x7ff1580c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ec98700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1580c8ec0 0x7ff1580701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ec98700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15806e060 con 0x7ff1580706f0 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ec98700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1580cede0 con 0x7ff1580c8ae0 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ec98700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15806fa40 con 0x7ff1580c8ec0 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ca34700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1580c8ae0 0x7ff1580c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15d235700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1580706f0 0x7ff158151ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33416/0 (socket says 172.21.15.33:33416) 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15d235700 1 -- 172.21.15.33:0/2245218553 learned_addr learned my addr 172.21.15.33:0/2245218553 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:17.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff157fff700 1 --2- 172.21.15.33:0/2245218553 >> v2:172.21.15.42:3300/0 conn(0x7ff1580c8ec0 0x7ff1580701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ca34700 1 -- 172.21.15.33:0/2245218553 >> v2:172.21.15.42:3300/0 conn(0x7ff1580c8ec0 msgr2=0x7ff1580701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ca34700 1 --2- 172.21.15.33:0/2245218553 >> v2:172.21.15.42:3300/0 conn(0x7ff1580c8ec0 0x7ff1580701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ca34700 1 -- 172.21.15.33:0/2245218553 >> v1:172.21.15.33:6789/0 conn(0x7ff1580706f0 legacy=0x7ff158151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff15ca34700 1 -- 172.21.15.33:0/2245218553 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1581586a0 con 0x7ff1580c8ae0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.996+0000 7ff1577fe700 1 -- 172.21.15.33:0/2245218553 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906133299 0 0) 0x7ff15806e060 con 0x7ff1580706f0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ca34700 1 --2- 172.21.15.33:0/2245218553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff1580c38a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff158071e50 tx=0x7ff14c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ca8b78f3ed4c08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff1577fe700 1 -- 172.21.15.33:0/2245218553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c011830 con 0x7ff1580c8ae0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff1577fe700 1 -- 172.21.15.33:0/2245218553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c011990 con 0x7ff1580c8ae0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff1577fe700 1 -- 172.21.15.33:0/2245218553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c011d20 con 0x7ff1580c8ae0 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 -- 172.21.15.33:0/2245218553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 msgr2=0x7ff1580c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 --2- 172.21.15.33:0/2245218553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff1580c38a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff158071e50 tx=0x7ff14c009b20 comp rx=0 tx=0).stop 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 -- 172.21.15.33:0/2245218553 shutdown_connections 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 --2- 172.21.15.33:0/2245218553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff1580c38a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 --2- 172.21.15.33:0/2245218553 >> v2:172.21.15.42:3300/0 conn(0x7ff1580c8ec0 0x7ff1580701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 -- 172.21.15.33:0/2245218553 >> 172.21.15.33:0/2245218553 conn(0x7ff1580cde70 msgr2=0x7ff1580ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:17.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 -- 172.21.15.33:0/2245218553 shutdown_connections 2024-05-08T23:41:17.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.997+0000 7ff15ec98700 1 -- 172.21.15.33:0/2245218553 wait complete. 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.999+0000 7ff15ec98700 1 Processor -- start 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:17.999+0000 7ff15ec98700 1 -- start start 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1580706f0 0x7ff15814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff15814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1580c8ec0 0x7ff1581467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15814e670 con 0x7ff1580c8ae0 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15814f090 con 0x7ff1580c8ec0 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ec98700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15814ead0 con 0x7ff1580706f0 2024-05-08T23:41:17.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff15814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff15814cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60822/0 (socket says 172.21.15.33:60822) 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 -- 172.21.15.33:0/3921676103 learned_addr learned my addr 172.21.15.33:0/3921676103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15ca34700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1580706f0 0x7ff15814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff15d235700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1580c8ec0 0x7ff1581467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 -- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1580706f0 msgr2=0x7ff15814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1580706f0 0x7ff15814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 -- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1580c8ec0 msgr2=0x7ff1581467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1580c8ec0 0x7ff1581467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0094a0 con 0x7ff1580c8ae0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.000+0000 7ff157fff700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff15814cf80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e870 tx=0x7ff14400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.001+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144009590 con 0x7ff1580c8ae0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.001+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14400ee70 con 0x7ff1580c8ae0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.001+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14400ca50 con 0x7ff1580c8ae0 2024-05-08T23:41:17.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.001+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15814edb0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.001+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1581556e0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.002+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff15814faf0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.002+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff14400cbb0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.002+0000 7ff1557fa700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13c06ac90 0x7ff13c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff1440893c0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff15ca34700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13c06ac90 0x7ff13c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff1557fa700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7ff13c073110 0x7ff13c0755b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff13c075c80 con 0x7ff13c073110 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14408b9a0 con 0x7ff1580c8ae0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff15d235700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7ff13c073110 0x7ff13c0755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff15ca34700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13c06ac90 0x7ff13c06d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0090b0 tx=0x7ff14c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.003+0000 7ff15d235700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7ff13c073110 0x7ff13c0755b0 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.004+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== osd.5 v2:172.21.15.42:6800/47100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff13c075c80 con 0x7ff13c073110 2024-05-08T23:41:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.020+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 --> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff158158f40 con 0x7ff13c073110 2024-05-08T23:41:18.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff1557fa700 1 -- 172.21.15.33:0/3921676103 <== osd.5 v2:172.21.15.42:6800/47100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff150007150 con 0x7ff13c073110 2024-05-08T23:41:18.017 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:18.017 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:18.017 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:18.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7ff13c073110 msgr2=0x7ff13c0755b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.42:6800/47100,v1:172.21.15.42:6801/47100] conn(0x7ff13c073110 0x7ff13c0755b0 crc :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13c06ac90 msgr2=0x7ff13c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff13c06ac90 0x7ff13c06d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0090b0 tx=0x7ff14c008030 comp rx=0 tx=0).stop 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 msgr2=0x7ff15814cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15ec98700 1 --2- 172.21.15.33:0/3921676103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1580c8ae0 0x7ff15814cf80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff14400e870 tx=0x7ff14400ec30 comp rx=0 tx=0).stop 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.021+0000 7ff15d235700 1 -- 172.21.15.33:0/3921676103 reap_dead start 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.022+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 shutdown_connections 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.022+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 >> 172.21.15.33:0/3921676103 conn(0x7ff1580cde70 msgr2=0x7ff1580c2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.022+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 shutdown_connections 2024-05-08T23:41:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.022+0000 7ff15ec98700 1 -- 172.21.15.33:0/3921676103 wait complete. 2024-05-08T23:41:18.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T23:41:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.173+0000 7fd344c8f700 1 Processor -- start 2024-05-08T23:41:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.173+0000 7fd344c8f700 1 -- start start 2024-05-08T23:41:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd344c8f700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd3400712f0 0x7fd34015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd344c8f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd344c8f700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400c2860 con 0x7fd340154540 2024-05-08T23:41:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd344c8f700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400c3540 con 0x7fd3400712f0 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd344c8f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400c8620 con 0x7fd34006d980 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd33e59c700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd340154540 0x7fd3400c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33418/0 (socket says 172.21.15.33:33418) 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd33e59c700 1 -- 172.21.15.33:0/2283310895 learned_addr learned my addr 172.21.15.33:0/2283310895 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd33dd9b700 1 --2- 172.21.15.33:0/2283310895 >> v2:172.21.15.42:3300/0 conn(0x7fd3400712f0 0x7fd34015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd33ed9d700 1 --2- 172.21.15.33:0/2283310895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.174+0000 7fd33d59a700 1 -- 172.21.15.33:0/2283310895 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881457941 0 0) 0x7fd3400c2860 con 0x7fd340154540 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33d59a700 1 -- 172.21.15.33:0/2283310895 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3280035b0 con 0x7fd340154540 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33ed9d700 1 -- 172.21.15.33:0/2283310895 >> v2:172.21.15.42:3300/0 conn(0x7fd3400712f0 msgr2=0x7fd34015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33ed9d700 1 --2- 172.21.15.33:0/2283310895 >> v2:172.21.15.42:3300/0 conn(0x7fd3400712f0 0x7fd34015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33ed9d700 1 -- 172.21.15.33:0/2283310895 >> v1:172.21.15.33:6789/0 conn(0x7fd340154540 legacy=0x7fd3400c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33ed9d700 1 -- 172.21.15.33:0/2283310895 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400c2ae0 con 0x7fd34006d980 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33ed9d700 1 --2- 172.21.15.33:0/2283310895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015de30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd338009d90 tx=0x7fd33800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2611685c69702a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33d59a700 1 -- 172.21.15.33:0/2283310895 <== mon.2 v2:172.21.15.42:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800e070 con 0x7fd34006d980 2024-05-08T23:41:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33d59a700 1 -- 172.21.15.33:0/2283310895 <== mon.2 v2:172.21.15.42:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800bb20 con 0x7fd34006d980 2024-05-08T23:41:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.175+0000 7fd33d59a700 1 -- 172.21.15.33:0/2283310895 <== mon.2 v2:172.21.15.42:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800be90 con 0x7fd34006d980 2024-05-08T23:41:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 -- 172.21.15.33:0/2283310895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 msgr2=0x7fd34015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 --2- 172.21.15.33:0/2283310895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015de30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd338009d90 tx=0x7fd33800b330 comp rx=0 tx=0).stop 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 -- 172.21.15.33:0/2283310895 shutdown_connections 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 --2- 172.21.15.33:0/2283310895 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015de30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 --2- 172.21.15.33:0/2283310895 >> v2:172.21.15.42:3300/0 conn(0x7fd3400712f0 0x7fd34015d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 -- 172.21.15.33:0/2283310895 >> 172.21.15.33:0/2283310895 conn(0x7fd34006ce60 msgr2=0x7fd34006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 -- 172.21.15.33:0/2283310895 shutdown_connections 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.176+0000 7fd344c8f700 1 -- 172.21.15.33:0/2283310895 wait complete. 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.177+0000 7fd344c8f700 1 Processor -- start 2024-05-08T23:41:18.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.177+0000 7fd344c8f700 1 -- start start 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340154540 0x7fd34015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401592b0 con 0x7fd3400712f0 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd340158860 con 0x7fd340154540 2024-05-08T23:41:18.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd344c8f700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400b2f60 con 0x7fd34006d980 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd33dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd33dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60824/0 (socket says 172.21.15.33:60824) 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd33dd9b700 1 -- 172.21.15.33:0/4100470081 learned_addr learned my addr 172.21.15.33:0/4100470081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd33e59c700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.178+0000 7fd33ed9d700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340154540 0x7fd34015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 -- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 msgr2=0x7fd34015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 -- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340154540 msgr2=0x7fd34015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340154540 0x7fd34015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 -- 172.21.15.33:0/4100470081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338009a70 con 0x7fd3400712f0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd33dd9b700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e9b0 tx=0x7fd33400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334009710 con 0x7fd3400712f0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33400ee50 con 0x7fd3400712f0 2024-05-08T23:41:18.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34015b660 con 0x7fd3400712f0 2024-05-08T23:41:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400ca50 con 0x7fd3400712f0 2024-05-08T23:41:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.179+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3401584f0 con 0x7fd3400712f0 2024-05-08T23:41:18.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.180+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3401589d0 con 0x7fd3400712f0 2024-05-08T23:41:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.181+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd33400cbb0 con 0x7fd3400712f0 2024-05-08T23:41:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.181+0000 7fd32effd700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 0x7fd32006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.181+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd334089760 con 0x7fd3400712f0 2024-05-08T23:41:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.181+0000 7fd33e59c700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 0x7fd32006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.182+0000 7fd33e59c700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 0x7fd32006d180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd330007fd0 tx=0x7fd330009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.186+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd334054870 con 0x7fd3400712f0 2024-05-08T23:41:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.406+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3400c2dc0 con 0x7fd32006acc0 2024-05-08T23:41:18.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.406+0000 7fd32effd700 1 -- 172.21.15.33:0/4100470081 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd330001d50 con 0x7fd32006acc0 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 msgr2=0x7fd32006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 0x7fd32006d180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd330007fd0 tx=0x7fd330009040 comp rx=0 tx=0).stop 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 msgr2=0x7fd34014c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e9b0 tx=0x7fd33400ecc0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 shutdown_connections 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd32006acc0 0x7fd32006d180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd340154540 0x7fd34015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd34006d980 0x7fd34015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 --2- 172.21.15.33:0/4100470081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3400712f0 0x7fd34014c9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 >> 172.21.15.33:0/4100470081 conn(0x7fd34006ce60 msgr2=0x7fd3400717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 shutdown_connections 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.409+0000 7fd344c8f700 1 -- 172.21.15.33:0/4100470081 wait complete. 2024-05-08T23:41:18.407 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T23:41:18.420 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T23:41:18.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.565+0000 7f3933fff700 1 Processor -- start 2024-05-08T23:41:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.565+0000 7f3933fff700 1 -- start start 2024-05-08T23:41:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3933fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934154660 0x7f39340724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3933fff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39340c8f40 0x7f39340729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3933fff700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3934154a40 con 0x7f3934070ee0 2024-05-08T23:41:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3933fff700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393415e4d0 con 0x7f39340c8f40 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3933fff700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393406b6e0 con 0x7f3934154660 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f39327fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39340c8f40 0x7f39340729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f39337fe700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3934070ee0 0x7f39340c7310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33420/0 (socket says 172.21.15.33:33420) 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f39337fe700 1 -- 172.21.15.33:0/4008548389 learned_addr learned my addr 172.21.15.33:0/4008548389 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3932ffd700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934154660 0x7f39340724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3931ffb700 1 -- 172.21.15.33:0/4008548389 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989247917 0 0) 0x7f392c001490 con 0x7f3934070ee0 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f3931ffb700 1 -- 172.21.15.33:0/4008548389 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f391c0035b0 con 0x7f3934070ee0 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f39327fc700 1 -- 172.21.15.33:0/4008548389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934154660 msgr2=0x7f39340724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.566+0000 7f39327fc700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934154660 0x7f39340724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f39327fc700 1 -- 172.21.15.33:0/4008548389 >> v1:172.21.15.33:6789/0 conn(0x7f3934070ee0 legacy=0x7f39340c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f39327fc700 1 -- 172.21.15.33:0/4008548389 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393415d8c0 con 0x7f39340c8f40 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f39327fc700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39340729e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3928009840 tx=0x7f3928009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a196c31edb82f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f3931ffb700 1 -- 172.21.15.33:0/4008548389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928011830 con 0x7f39340c8f40 2024-05-08T23:41:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f3931ffb700 1 -- 172.21.15.33:0/4008548389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3928011990 con 0x7f39340c8f40 2024-05-08T23:41:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f3931ffb700 1 -- 172.21.15.33:0/4008548389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928011d20 con 0x7f39340c8f40 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f3933fff700 1 -- 172.21.15.33:0/4008548389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 msgr2=0x7f39340729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.567+0000 7f3933fff700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39340729e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3928009840 tx=0x7f3928009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 -- 172.21.15.33:0/4008548389 shutdown_connections 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934154660 0x7f39340724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 --2- 172.21.15.33:0/4008548389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39340729e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 -- 172.21.15.33:0/4008548389 >> 172.21.15.33:0/4008548389 conn(0x7f39340cc910 msgr2=0x7f39340ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 -- 172.21.15.33:0/4008548389 shutdown_connections 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.568+0000 7f3933fff700 1 -- 172.21.15.33:0/4008548389 wait complete. 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.570+0000 7f3933fff700 1 Processor -- start 2024-05-08T23:41:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.570+0000 7f3933fff700 1 -- start start 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934070ee0 0x7f39341f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39341f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934154660 0x7f3934152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934156500 con 0x7f39340c8f40 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393414fa90 con 0x7f3934154660 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3933fff700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3934157650 con 0x7f3934070ee0 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39341f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39341f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60852/0 (socket says 172.21.15.33:60852) 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 -- 172.21.15.33:0/1488351865 learned_addr learned my addr 172.21.15.33:0/1488351865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f3932ffd700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934070ee0 0x7f39341f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39337fe700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934154660 0x7f3934152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 -- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934070ee0 msgr2=0x7f39341f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3934070ee0 0x7f39341f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 -- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934154660 msgr2=0x7f3934152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3934154660 0x7f3934152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280094a0 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.571+0000 7f39327fc700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39341f31a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39280090b0 tx=0x7f3928004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280036a0 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392801f070 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393414fcc0 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928003a00 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3934157b60 con 0x7f39340c8f40 2024-05-08T23:41:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.572+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f393415e160 con 0x7f39340c8f40 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.573+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3928003b60 con 0x7f39340c8f40 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.573+0000 7f391b7fe700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f392006ac90 0x7f392006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3928021070 con 0x7f39340c8f40 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f3932ffd700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f392006ac90 0x7f392006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f391b7fe700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7f3920073110 0x7f39200755b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3920075c80 con 0x7f3920073110 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f392808f2c0 con 0x7f39340c8f40 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f39337fe700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7f3920073110 0x7f39200755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f3932ffd700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f392006ac90 0x7f392006d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3924009ad0 tx=0x7f3924008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.574+0000 7f39337fe700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7f3920073110 0x7f39200755b0 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.575+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== osd.6 v2:172.21.15.42:6816/47123 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3920075c80 con 0x7f3920073110 2024-05-08T23:41:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.591+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 --> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f393415de80 con 0x7f3920073110 2024-05-08T23:41:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.591+0000 7f391b7fe700 1 -- 172.21.15.33:0/1488351865 <== osd.6 v2:172.21.15.42:6816/47123 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f392c0070e0 con 0x7f3920073110 2024-05-08T23:41:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:18.588 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7f3920073110 msgr2=0x7f39200755b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.42:6816/47123,v1:172.21.15.42:6817/47123] conn(0x7f3920073110 0x7f39200755b0 crc :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f392006ac90 msgr2=0x7f392006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f392006ac90 0x7f392006d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3924009ad0 tx=0x7f3924008040 comp rx=0 tx=0).stop 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 msgr2=0x7f39341f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 --2- 172.21.15.33:0/1488351865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39340c8f40 0x7f39341f31a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f39280090b0 tx=0x7f3928004750 comp rx=0 tx=0).stop 2024-05-08T23:41:18.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f39337fe700 1 -- 172.21.15.33:0/1488351865 reap_dead start 2024-05-08T23:41:18.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 shutdown_connections 2024-05-08T23:41:18.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 >> 172.21.15.33:0/1488351865 conn(0x7f39340cc910 msgr2=0x7f39340c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 shutdown_connections 2024-05-08T23:41:18.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.592+0000 7f3933fff700 1 -- 172.21.15.33:0/1488351865 wait complete. 2024-05-08T23:41:18.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T23:41:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.744+0000 7f94f78e8700 1 Processor -- start 2024-05-08T23:41:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.744+0000 7f94f78e8700 1 -- start start 2024-05-08T23:41:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f01522f0 0x7f94f00c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f94f0070990 0x7f94f00cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94f00b3a30 0x7f94f00ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f0070780 con 0x7f94f00b3a30 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f00c3b10 con 0x7f94f0070990 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f78e8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c8db0 con 0x7f94f01522f0 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f5e85700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94f00b3a30 0x7f94f00ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f5e85700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94f00b3a30 0x7f94f00ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60866/0 (socket says 172.21.15.33:60866) 2024-05-08T23:41:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f5e85700 1 -- 172.21.15.33:0/1925933105 learned_addr learned my addr 172.21.15.33:0/1925933105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f5684700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f01522f0 0x7f94f00c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.745+0000 7f94f4e83700 1 --2- 172.21.15.33:0/1925933105 >> v2:172.21.15.42:3300/0 conn(0x7f94f0070990 0x7f94f00cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 -- 172.21.15.33:0/1925933105 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f01522f0 msgr2=0x7f94f00c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f01522f0 0x7f94f00c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 -- 172.21.15.33:0/1925933105 >> v2:172.21.15.42:3300/0 conn(0x7f94f0070990 msgr2=0x7f94f00cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 --2- 172.21.15.33:0/1925933105 >> v2:172.21.15.42:3300/0 conn(0x7f94f0070990 0x7f94f00cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 -- 172.21.15.33:0/1925933105 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0070200 con 0x7f94f00b3a30 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94f5e85700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f00b3a30 0x7f94f00ce300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94f0072d70 tx=0x7f94e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce8d0aa5e15ecde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94e7fff700 1 -- 172.21.15.33:0/1925933105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800e070 con 0x7f94f00b3a30 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94e7fff700 1 -- 172.21.15.33:0/1925933105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e800b910 con 0x7f94f00b3a30 2024-05-08T23:41:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.746+0000 7f94e7fff700 1 -- 172.21.15.33:0/1925933105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800bcc0 con 0x7f94f00b3a30 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 -- 172.21.15.33:0/1925933105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f00b3a30 msgr2=0x7f94f00ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f00b3a30 0x7f94f00ce300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94f0072d70 tx=0x7f94e800b330 comp rx=0 tx=0).stop 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 -- 172.21.15.33:0/1925933105 shutdown_connections 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f00b3a30 0x7f94f00ce300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 --2- 172.21.15.33:0/1925933105 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f01522f0 0x7f94f00c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 --2- 172.21.15.33:0/1925933105 >> v2:172.21.15.42:3300/0 conn(0x7f94f0070990 0x7f94f00cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 -- 172.21.15.33:0/1925933105 >> 172.21.15.33:0/1925933105 conn(0x7f94f0072230 msgr2=0x7f94f0072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 -- 172.21.15.33:0/1925933105 shutdown_connections 2024-05-08T23:41:18.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.747+0000 7f94f78e8700 1 -- 172.21.15.33:0/1925933105 wait complete. 2024-05-08T23:41:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.748+0000 7f94f78e8700 1 Processor -- start 2024-05-08T23:41:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.748+0000 7f94f78e8700 1 -- start start 2024-05-08T23:41:18.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f00b3a30 0x7f94f006ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94f01522f0 0x7f94f01f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0159bf0 con 0x7f94f0070990 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0149250 con 0x7f94f01522f0 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f78e8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c53f0 con 0x7f94f00b3a30 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60882/0 (socket says 172.21.15.33:60882) 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 -- 172.21.15.33:0/3760525731 learned_addr learned my addr 172.21.15.33:0/3760525731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5e85700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94f01522f0 0x7f94f01f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f4e83700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f00b3a30 0x7f94f006ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 -- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f00b3a30 msgr2=0x7f94f006ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f00b3a30 0x7f94f006ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 -- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94f01522f0 msgr2=0x7f94f01f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94f01522f0 0x7f94f01f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.749+0000 7f94f5684700 1 -- 172.21.15.33:0/3760525731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8009b30 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94f5684700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e870 tx=0x7f94ec00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec009590 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec013680 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f01f3b10 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0139f0 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.750+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f006c910 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.751+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f01f3d50 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.752+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f94ec013b50 con 0x7f94f0070990 2024-05-08T23:41:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.752+0000 7f94e5ffb700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 0x7f94d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:18.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.752+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f94ec0892f0 con 0x7f94f0070990 2024-05-08T23:41:18.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.752+0000 7f94f4e83700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 0x7f94d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:18.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.753+0000 7f94f4e83700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 0x7f94d406d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f94f00b33c0 tx=0x7f94e0009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:18.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.757+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f94ec054360 con 0x7f94f0070990 2024-05-08T23:41:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.977+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f94f00c3310 con 0x7f94d406ac90 2024-05-08T23:41:18.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.978+0000 7f94e5ffb700 1 -- 172.21.15.33:0/3760525731 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f94e0004020 con 0x7f94d406ac90 2024-05-08T23:41:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 msgr2=0x7f94d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 0x7f94d406d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f94f00b33c0 tx=0x7f94e0009210 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 msgr2=0x7f94f0148d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e870 tx=0x7f94ec00eb80 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 shutdown_connections 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f94d406ac90 0x7f94d406d150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94f01522f0 0x7f94f01f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f0070990 0x7f94f0148d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 --2- 172.21.15.33:0/3760525731 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f94f00b3a30 0x7f94f006ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 >> 172.21.15.33:0/3760525731 conn(0x7f94f0072230 msgr2=0x7f94f014dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 shutdown_connections 2024-05-08T23:41:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:18.980+0000 7f94f78e8700 1 -- 172.21.15.33:0/3760525731 wait complete. 2024-05-08T23:41:18.980 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T23:41:18.991 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T23:41:18.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T23:41:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.151+0000 7f47f6948700 1 Processor -- start 2024-05-08T23:41:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.151+0000 7f47f6948700 1 -- start start 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f47f015b100 0x7f47f015e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47f015a330 0x7f47f00707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f00c69c0 0x7f47f006ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f015ddf0 con 0x7f47f015a330 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f00c77c0 con 0x7f47f015b100 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47f6948700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f015daa0 con 0x7f47f00c69c0 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.152+0000 7f47ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47f015a330 0x7f47f00707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47f015a330 0x7f47f00707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60892/0 (socket says 172.21.15.33:60892) 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 -- 172.21.15.33:0/2456075746 learned_addr learned my addr 172.21.15.33:0/2456075746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47effff700 1 --2- 172.21.15.33:0/2456075746 >> v2:172.21.15.42:3300/0 conn(0x7f47f015b100 0x7f47f015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47f4ee5700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f00c69c0 0x7f47f006ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 -- 172.21.15.33:0/2456075746 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f00c69c0 msgr2=0x7f47f006ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f00c69c0 0x7f47f006ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 -- 172.21.15.33:0/2456075746 >> v2:172.21.15.42:3300/0 conn(0x7f47f015b100 msgr2=0x7f47f015e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/2456075746 >> v2:172.21.15.42:3300/0 conn(0x7f47f015b100 0x7f47f015e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 -- 172.21.15.33:0/2456075746 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00c6f20 con 0x7f47f015a330 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f00707b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47f00c9c60 tx=0x7f47e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b7f93d61bb4ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47eeffd700 1 -- 172.21.15.33:0/2456075746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011830 con 0x7f47f015a330 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47eeffd700 1 -- 172.21.15.33:0/2456075746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e4011990 con 0x7f47f015a330 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.153+0000 7f47eeffd700 1 -- 172.21.15.33:0/2456075746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4011d20 con 0x7f47f015a330 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 -- 172.21.15.33:0/2456075746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 msgr2=0x7f47f00707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f00707b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f47f00c9c60 tx=0x7f47e4009bd0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 -- 172.21.15.33:0/2456075746 shutdown_connections 2024-05-08T23:41:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f00c69c0 0x7f47f006ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 --2- 172.21.15.33:0/2456075746 >> v2:172.21.15.42:3300/0 conn(0x7f47f015b100 0x7f47f015e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 --2- 172.21.15.33:0/2456075746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f00707b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 -- 172.21.15.33:0/2456075746 >> 172.21.15.33:0/2456075746 conn(0x7f47f0066f50 msgr2=0x7f47f0067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 -- 172.21.15.33:0/2456075746 shutdown_connections 2024-05-08T23:41:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.154+0000 7f47f6948700 1 -- 172.21.15.33:0/2456075746 wait complete. 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.155+0000 7f47f6948700 1 Processor -- start 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.155+0000 7f47f6948700 1 -- start start 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f00c69c0 0x7f47f01e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f01f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f01f6630 0x7f47f00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00c4e00 con 0x7f47f015a330 2024-05-08T23:41:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01e6470 con 0x7f47f00c69c0 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f6948700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00c5360 con 0x7f47f01f6630 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f01f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f01f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60904/0 (socket says 172.21.15.33:60904) 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47ef7fe700 1 -- 172.21.15.33:0/171966297 learned_addr learned my addr 172.21.15.33:0/171966297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47effff700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f00c69c0 0x7f47f01e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.156+0000 7f47f4ee5700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f01f6630 0x7f47f00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 -- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f01f6630 msgr2=0x7f47f00c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f47f01f6630 0x7f47f00c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 -- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f00c69c0 msgr2=0x7f47f01e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f00c69c0 0x7f47f01e58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4009550 con 0x7f47f015a330 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ef7fe700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f01f60f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009c10 tx=0x7f47e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4004240 con 0x7f47f015a330 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e401f070 con 0x7f47f015a330 2024-05-08T23:41:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f01f6a90 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40036a0 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.157+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f00c3660 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.158+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47f006b440 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.158+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f47e4021030 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47ecff9700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f47dc06acc0 0x7f47dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f47e408a4c0 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47effff700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f47dc06acc0 0x7f47dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47ecff9700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7f47dc073140 0x7f47dc0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47dc075cb0 con 0x7f47dc073140 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e408cb80 con 0x7f47f015a330 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47f4ee5700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7f47dc073140 0x7f47dc0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.159+0000 7f47effff700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f47dc06acc0 0x7f47dc06d180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009be0 tx=0x7f47e0009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.160+0000 7f47f4ee5700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7f47dc073140 0x7f47dc0755e0 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.161+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== osd.7 v2:172.21.15.42:6808/47101 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f47dc075cb0 con 0x7f47dc073140 2024-05-08T23:41:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.176+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 --> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47f01f5f80 con 0x7f47dc073140 2024-05-08T23:41:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47ecff9700 1 -- 172.21.15.33:0/171966297 <== osd.7 v2:172.21.15.42:6808/47101 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f47f01f5f80 con 0x7f47dc073140 2024-05-08T23:41:19.174 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T23:41:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T23:41:19.174 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T23:41:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7f47dc073140 msgr2=0x7f47dc0755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.42:6808/47101,v1:172.21.15.42:6809/47101] conn(0x7f47dc073140 0x7f47dc0755e0 crc :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f47dc06acc0 msgr2=0x7f47dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f47dc06acc0 0x7f47dc06d180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009be0 tx=0x7f47e0009210 comp rx=0 tx=0).stop 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 msgr2=0x7f47f01f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 --2- 172.21.15.33:0/171966297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015a330 0x7f47f01f60f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009c10 tx=0x7f47e4004750 comp rx=0 tx=0).stop 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f4ee5700 1 -- 172.21.15.33:0/171966297 reap_dead start 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 shutdown_connections 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 >> 172.21.15.33:0/171966297 conn(0x7f47f0066f50 msgr2=0x7f47f015d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 shutdown_connections 2024-05-08T23:41:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.177+0000 7f47f6948700 1 -- 172.21.15.33:0/171966297 wait complete. 2024-05-08T23:41:19.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T23:41:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.329+0000 7ff615e64700 1 Processor -- start 2024-05-08T23:41:19.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.329+0000 7ff615e64700 1 -- start start 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.330+0000 7ff615e64700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 0x7ff6100c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.330+0000 7ff615e64700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff610157a90 0x7ff610150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.330+0000 7ff615e64700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6100c8610 con 0x7ff6100c9280 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.330+0000 7ff615e64700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100c4470 con 0x7ff610157a90 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.330+0000 7ff615e64700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100c7d90 con 0x7ff6100c8ea0 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff610157a90 0x7ff610150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff610157a90 0x7ff610150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60908/0 (socket says 172.21.15.33:60908) 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 -- 172.21.15.33:0/2807868849 learned_addr learned my addr 172.21.15.33:0/2807868849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60f7fe700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 0x7ff6100c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 -- 172.21.15.33:0/2807868849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 msgr2=0x7ff6100c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 0x7ff6100c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 -- 172.21.15.33:0/2807868849 >> v1:172.21.15.42:6789/0 conn(0x7ff6100c9280 legacy=0x7ff610071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 -- 172.21.15.33:0/2807868849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610157ef0 con 0x7ff610157a90 2024-05-08T23:41:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60f7fe700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 0x7ff6100c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60ffff700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff610157a90 0x7ff610150d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff6100c7ed0 tx=0x7ff60800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bf060ab151c96e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60e7fc700 1 -- 172.21.15.33:0/2807868849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e070 con 0x7ff610157a90 2024-05-08T23:41:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60e7fc700 1 -- 172.21.15.33:0/2807868849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608003a20 con 0x7ff610157a90 2024-05-08T23:41:19.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.331+0000 7ff60e7fc700 1 -- 172.21.15.33:0/2807868849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608003dd0 con 0x7ff610157a90 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 -- 172.21.15.33:0/2807868849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff610157a90 msgr2=0x7ff610150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff610157a90 0x7ff610150d00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff6100c7ed0 tx=0x7ff60800b330 comp rx=0 tx=0).stop 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 -- 172.21.15.33:0/2807868849 shutdown_connections 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff610157a90 0x7ff610150d00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 --2- 172.21.15.33:0/2807868849 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff6100c8ea0 0x7ff6100c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 -- 172.21.15.33:0/2807868849 >> 172.21.15.33:0/2807868849 conn(0x7ff61006a990 msgr2=0x7ff61006ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 -- 172.21.15.33:0/2807868849 shutdown_connections 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.332+0000 7ff615e64700 1 -- 172.21.15.33:0/2807868849 wait complete. 2024-05-08T23:41:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.333+0000 7ff615e64700 1 Processor -- start 2024-05-08T23:41:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.333+0000 7ff615e64700 1 -- start start 2024-05-08T23:41:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 0x7ff61014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff610157a90 0x7ff6100c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61014a670 con 0x7ff6100c8ea0 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100cb880 con 0x7ff6100c9280 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff615e64700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101e2e10 con 0x7ff610157a90 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff60f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff60f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60920/0 (socket says 172.21.15.33:60920) 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.334+0000 7ff60f7fe700 1 -- 172.21.15.33:0/1710277818 learned_addr learned my addr 172.21.15.33:0/1710277818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60ffff700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff610157a90 0x7ff6100c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60effd700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 0x7ff61014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 -- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff610157a90 msgr2=0x7ff6100c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff610157a90 0x7ff6100c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 -- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 msgr2=0x7ff61014fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 0x7ff61014fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 -- 172.21.15.33:0/1710277818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009b30 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60effd700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 0x7ff61014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff60f7fe700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e810 tx=0x7ff60000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600004230 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600004390 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000036a0 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610071450 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.335+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61014a390 con 0x7ff6100c8ea0 2024-05-08T23:41:19.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.336+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610075460 con 0x7ff6100c8ea0 2024-05-08T23:41:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.337+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff600003800 con 0x7ff6100c8ea0 2024-05-08T23:41:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.337+0000 7ff607fff700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 0x7ff5f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.337+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7ff600088e80 con 0x7ff6100c8ea0 2024-05-08T23:41:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.337+0000 7ff60effd700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 0x7ff5f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.338+0000 7ff60effd700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 0x7ff5f006d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff6101e3980 tx=0x7ff5f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.342+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff600053f90 con 0x7ff6100c8ea0 2024-05-08T23:41:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.561+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6100c72b0 con 0x7ff5f006ac90 2024-05-08T23:41:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.561+0000 7ff607fff700 1 -- 172.21.15.33:0/1710277818 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff5f8001c40 con 0x7ff5f006ac90 2024-05-08T23:41:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 msgr2=0x7ff5f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 0x7ff5f006d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff6101e3980 tx=0x7ff5f8009040 comp rx=0 tx=0).stop 2024-05-08T23:41:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 msgr2=0x7ff61014fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff60000e810 tx=0x7ff60000ebd0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 shutdown_connections 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff5f006ac90 0x7ff5f006d150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff610157a90 0x7ff6100c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6100c8ea0 0x7ff61014fab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 --2- 172.21.15.33:0/1710277818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6100c9280 0x7ff61014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 >> 172.21.15.33:0/1710277818 conn(0x7ff61006a990 msgr2=0x7ff61014ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 shutdown_connections 2024-05-08T23:41:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.564+0000 7ff615e64700 1 -- 172.21.15.33:0/1710277818 wait complete. 2024-05-08T23:41:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T23:41:19.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:41:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.716+0000 7f2e5d2e7700 1 Processor -- start 2024-05-08T23:41:19.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.716+0000 7f2e5d2e7700 1 -- start start 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e5d2e7700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 0x7f2e5815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e5d2e7700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e58070990 0x7f2e581541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e5d2e7700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5815a930 con 0x7f2e580ce410 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e5d2e7700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5815ac10 con 0x7f2e58070990 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e5d2e7700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5815aef0 con 0x7f2e580ce030 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e58070990 0x7f2e581541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e58070990 0x7f2e581541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60932/0 (socket says 172.21.15.33:60932) 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 -- 172.21.15.33:0/4235403773 learned_addr learned my addr 172.21.15.33:0/4235403773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e56ffd700 1 --2- 172.21.15.33:0/4235403773 >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 0x7f2e5815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 -- 172.21.15.33:0/4235403773 >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 msgr2=0x7f2e5815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 --2- 172.21.15.33:0/4235403773 >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 0x7f2e5815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 -- 172.21.15.33:0/4235403773 >> v1:172.21.15.42:6789/0 conn(0x7f2e580ce410 legacy=0x7f2e580c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 -- 172.21.15.33:0/4235403773 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580c8fb0 con 0x7f2e58070990 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e56ffd700 1 --2- 172.21.15.33:0/4235403773 >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 0x7f2e5815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.717+0000 7f2e567fc700 1 --2- 172.21.15.33:0/4235403773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e581541a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e580730c0 tx=0x7f2e4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4b33e5734ee452 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e55ffb700 1 -- 172.21.15.33:0/4235403773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00b870 con 0x7f2e58070990 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e55ffb700 1 -- 172.21.15.33:0/4235403773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c00b9d0 con 0x7f2e58070990 2024-05-08T23:41:19.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e55ffb700 1 -- 172.21.15.33:0/4235403773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00bd60 con 0x7f2e58070990 2024-05-08T23:41:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/4235403773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 msgr2=0x7f2e581541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/4235403773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e581541a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e580730c0 tx=0x7f2e4c009bd0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/4235403773 shutdown_connections 2024-05-08T23:41:19.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/4235403773 >> v2:172.21.15.42:3300/0 conn(0x7f2e580ce030 0x7f2e5815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/4235403773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e581541a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/4235403773 >> 172.21.15.33:0/4235403773 conn(0x7f2e58072230 msgr2=0x7f2e58072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/4235403773 shutdown_connections 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.718+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/4235403773 wait complete. 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.720+0000 7f2e5d2e7700 1 Processor -- start 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.720+0000 7f2e5d2e7700 1 -- start start 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e580ce030 0x7f2e580af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 0x7f2e581541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581efba0 con 0x7f2e58070990 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581efe40 con 0x7f2e580ce030 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581f0120 con 0x7f2e580ce410 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60944/0 (socket says 172.21.15.33:60944) 2024-05-08T23:41:19.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 -- 172.21.15.33:0/2651040902 learned_addr learned my addr 172.21.15.33:0/2651040902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e567fc700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e580ce030 0x7f2e580af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e577fe700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 0x7f2e581541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 -- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 msgr2=0x7f2e581541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 0x7f2e581541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 -- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e580ce030 msgr2=0x7f2e580af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e580ce030 0x7f2e580af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 -- 172.21.15.33:0/2651040902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c009550 con 0x7f2e58070990 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e577fe700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 0x7f2e581541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e56ffd700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800e810 tx=0x7f2e4800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48004230 con 0x7f2e58070990 2024-05-08T23:41:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e480043b0 con 0x7f2e58070990 2024-05-08T23:41:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e581f0400 con 0x7f2e58070990 2024-05-08T23:41:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.721+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480038e0 con 0x7f2e58070990 2024-05-08T23:41:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.722+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e581f09c0 con 0x7f2e58070990 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.722+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58075460 con 0x7f2e58070990 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.723+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f2e48003a40 con 0x7f2e58070990 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.723+0000 7f2e3f7fe700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 0x7f2e4406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.724+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f2e48088e10 con 0x7f2e58070990 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.724+0000 7f2e567fc700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 0x7f2e4406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.724+0000 7f2e567fc700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 0x7f2e4406d180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2e58072010 tx=0x7f2e4c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:19.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.728+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f2e48053f20 con 0x7f2e58070990 2024-05-08T23:41:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.942+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e5815b1d0 con 0x7f2e4406acc0 2024-05-08T23:41:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.950+0000 7f2e3f7fe700 1 -- 172.21.15.33:0/2651040902 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288135 (secure 0 0 0) 0x7f2e4c00bb40 con 0x7f2e4406acc0 2024-05-08T23:41:19.951 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 msgr2=0x7f2e4406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 0x7f2e4406d180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2e58072010 tx=0x7f2e4c008030 comp rx=0 tx=0).stop 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 msgr2=0x7f2e58157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800e810 tx=0x7f2e4800eb20 comp rx=0 tx=0).stop 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 shutdown_connections 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f2e4406acc0 0x7f2e4406d180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f2e580ce410 0x7f2e581541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58070990 0x7f2e58157940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 --2- 172.21.15.33:0/2651040902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e580ce030 0x7f2e580af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 >> 172.21.15.33:0/2651040902 conn(0x7f2e58072230 msgr2=0x7f2e58151dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 shutdown_connections 2024-05-08T23:41:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:19.954+0000 7f2e5d2e7700 1 -- 172.21.15.33:0/2651040902 wait complete. 2024-05-08T23:41:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:41:19.965 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-08T23:41:19.496400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17884105,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320951,"num_write_kb":49019247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422187,"ondisk_log_size":422187,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4065176,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4028211,"kb_avail":750909544,"statfs":{"total":773094113280,"available":768931373056,"internally_reserved":0,"allocated":37773312,"data_stored":36472434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4124888632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"3.3f","version":"23'5702","reported_seq":5718,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.136862+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:41:12.136862+0000","last_peered":"2024-05-08T23:41:12.136862+0000","last_clean":"2024-05-08T23:41:12.136862+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:41:12.136862+0000","last_undegraded":"2024-05-08T23:41:12.136862+0000","last_fullsized":"2024-05-08T23:41:12.136862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:37.034688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8392","reported_seq":8409,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040666+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:41:12.040666+0000","last_peered":"2024-05-08T23:41:12.040666+0000","last_clean":"2024-05-08T23:41:12.040666+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:41:12.040666+0000","last_undegraded":"2024-05-08T23:41:12.040666+0000","last_fullsized":"2024-05-08T23:41:12.040666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6514","reported_seq":6531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.038998+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:41:12.038998+0000","last_peered":"2024-05-08T23:41:12.038998+0000","last_clean":"2024-05-08T23:41:12.038998+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:41:12.038998+0000","last_undegraded":"2024-05-08T23:41:12.038998+0000","last_fullsized":"2024-05-08T23:41:12.038998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5995","reported_seq":6012,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.086784+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:41:12.086784+0000","last_peered":"2024-05-08T23:41:12.086784+0000","last_clean":"2024-05-08T23:41:12.086784+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:41:12.086784+0000","last_undegraded":"2024-05-08T23:41:12.086784+0000","last_fullsized":"2024-05-08T23:41:12.086784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7607","reported_seq":7621,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.030041+0000","last_change":"2024-05-08T23:32:59.855454+0000","last_active":"2024-05-08T23:41:12.030041+0000","last_peered":"2024-05-08T23:41:12.030041+0000","last_clean":"2024-05-08T23:41:12.030041+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:41:12.030041+0000","last_undegraded":"2024-05-08T23:41:12.030041+0000","last_fullsized":"2024-05-08T23:41:12.030041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5753","reported_seq":5769,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.172968+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:41:12.172968+0000","last_peered":"2024-05-08T23:41:12.172968+0000","last_clean":"2024-05-08T23:41:12.172968+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:41:12.172968+0000","last_undegraded":"2024-05-08T23:41:12.172968+0000","last_fullsized":"2024-05-08T23:41:12.172968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5753,"log_dups_size":0,"ondisk_log_size":5753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6497","reported_seq":6512,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051286+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:41:12.051286+0000","last_peered":"2024-05-08T23:41:12.051286+0000","last_clean":"2024-05-08T23:41:12.051286+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:41:12.051286+0000","last_undegraded":"2024-05-08T23:41:12.051286+0000","last_fullsized":"2024-05-08T23:41:12.051286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.315170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6893","reported_seq":6909,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.108812+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:41:12.108812+0000","last_peered":"2024-05-08T23:41:12.108812+0000","last_clean":"2024-05-08T23:41:12.108812+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:41:12.108812+0000","last_undegraded":"2024-05-08T23:41:12.108812+0000","last_fullsized":"2024-05-08T23:41:12.108812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8092","reported_seq":8107,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.162769+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:41:12.162769+0000","last_peered":"2024-05-08T23:41:12.162769+0000","last_clean":"2024-05-08T23:41:12.162769+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:41:12.162769+0000","last_undegraded":"2024-05-08T23:41:12.162769+0000","last_fullsized":"2024-05-08T23:41:12.162769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8092,"log_dups_size":0,"ondisk_log_size":8092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:35.601253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7426","reported_seq":7441,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.158966+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:41:12.158966+0000","last_peered":"2024-05-08T23:41:12.158966+0000","last_clean":"2024-05-08T23:41:12.158966+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:41:12.158966+0000","last_undegraded":"2024-05-08T23:41:12.158966+0000","last_fullsized":"2024-05-08T23:41:12.158966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7426,"log_dups_size":0,"ondisk_log_size":7426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:04:30.380295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6322","reported_seq":6337,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.012886+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:41:12.012886+0000","last_peered":"2024-05-08T23:41:12.012886+0000","last_clean":"2024-05-08T23:41:12.012886+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:41:12.012886+0000","last_undegraded":"2024-05-08T23:41:12.012886+0000","last_fullsized":"2024-05-08T23:41:12.012886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:14.856251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5509","reported_seq":5524,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.136136+0000","last_change":"2024-05-08T23:32:10.352427+0000","last_active":"2024-05-08T23:41:12.136136+0000","last_peered":"2024-05-08T23:41:12.136136+0000","last_clean":"2024-05-08T23:41:12.136136+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:41:12.136136+0000","last_undegraded":"2024-05-08T23:41:12.136136+0000","last_fullsized":"2024-05-08T23:41:12.136136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:52.847575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6039","reported_seq":6055,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.042831+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:41:12.042831+0000","last_peered":"2024-05-08T23:41:12.042831+0000","last_clean":"2024-05-08T23:41:12.042831+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:41:12.042831+0000","last_undegraded":"2024-05-08T23:41:12.042831+0000","last_fullsized":"2024-05-08T23:41:12.042831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6680","reported_seq":6696,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.060349+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:41:12.060349+0000","last_peered":"2024-05-08T23:41:12.060349+0000","last_clean":"2024-05-08T23:41:12.060349+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:41:12.060349+0000","last_undegraded":"2024-05-08T23:41:12.060349+0000","last_fullsized":"2024-05-08T23:41:12.060349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6680,"log_dups_size":0,"ondisk_log_size":6680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6521","reported_seq":6537,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.080846+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:41:12.080846+0000","last_peered":"2024-05-08T23:41:12.080846+0000","last_clean":"2024-05-08T23:41:12.080846+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:41:12.080846+0000","last_undegraded":"2024-05-08T23:41:12.080846+0000","last_fullsized":"2024-05-08T23:41:12.080846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6521,"log_dups_size":0,"ondisk_log_size":6521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5781","reported_seq":5794,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.071902+0000","last_change":"2024-05-08T23:32:59.856316+0000","last_active":"2024-05-08T23:41:12.071902+0000","last_peered":"2024-05-08T23:41:12.071902+0000","last_clean":"2024-05-08T23:41:12.071902+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:41:12.071902+0000","last_undegraded":"2024-05-08T23:41:12.071902+0000","last_fullsized":"2024-05-08T23:41:12.071902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:19.763425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7981","reported_seq":7997,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.038955+0000","last_change":"2024-05-08T23:32:10.349297+0000","last_active":"2024-05-08T23:41:12.038955+0000","last_peered":"2024-05-08T23:41:12.038955+0000","last_clean":"2024-05-08T23:41:12.038955+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:41:12.038955+0000","last_undegraded":"2024-05-08T23:41:12.038955+0000","last_fullsized":"2024-05-08T23:41:12.038955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5310","reported_seq":5325,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.077008+0000","last_change":"2024-05-08T23:32:10.348102+0000","last_active":"2024-05-08T23:41:12.077008+0000","last_peered":"2024-05-08T23:41:12.077008+0000","last_clean":"2024-05-08T23:41:12.077008+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:41:12.077008+0000","last_undegraded":"2024-05-08T23:41:12.077008+0000","last_fullsized":"2024-05-08T23:41:12.077008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:05:39.609288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6330","reported_seq":6345,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.088191+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:41:12.088191+0000","last_peered":"2024-05-08T23:41:12.088191+0000","last_clean":"2024-05-08T23:41:12.088191+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:41:12.088191+0000","last_undegraded":"2024-05-08T23:41:12.088191+0000","last_fullsized":"2024-05-08T23:41:12.088191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6330,"log_dups_size":0,"ondisk_log_size":6330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:31.033168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'383","reported_seq":401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.883984+0000","last_change":"2024-05-08T23:32:09.892147+0000","last_active":"2024-05-08T23:40:50.883984+0000","last_peered":"2024-05-08T23:40:50.883984+0000","last_clean":"2024-05-08T23:40:50.883984+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:40:50.883984+0000","last_undegraded":"2024-05-08T23:40:50.883984+0000","last_fullsized":"2024-05-08T23:40:50.883984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6847","reported_seq":6861,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.015498+0000","last_change":"2024-05-08T23:33:00.026361+0000","last_active":"2024-05-08T23:41:12.015498+0000","last_peered":"2024-05-08T23:41:12.015498+0000","last_clean":"2024-05-08T23:41:12.015498+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:41:12.015498+0000","last_undegraded":"2024-05-08T23:41:12.015498+0000","last_fullsized":"2024-05-08T23:41:12.015498+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:35:26.768597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.864160+0000","last_change":"2024-05-08T23:32:09.885107+0000","last_active":"2024-05-08T23:40:50.864160+0000","last_peered":"2024-05-08T23:40:50.864160+0000","last_clean":"2024-05-08T23:40:50.864160+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:40:50.864160+0000","last_undegraded":"2024-05-08T23:40:50.864160+0000","last_fullsized":"2024-05-08T23:40:50.864160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7917","reported_seq":7942,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.069649+0000","last_change":"2024-05-08T23:32:59.859469+0000","last_active":"2024-05-08T23:41:12.069649+0000","last_peered":"2024-05-08T23:41:12.069649+0000","last_clean":"2024-05-08T23:41:12.069649+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:41:12.069649+0000","last_undegraded":"2024-05-08T23:41:12.069649+0000","last_fullsized":"2024-05-08T23:41:12.069649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'10","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.837769+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:40:50.837769+0000","last_peered":"2024-05-08T23:40:50.837769+0000","last_clean":"2024-05-08T23:40:50.837769+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:40:50.837769+0000","last_undegraded":"2024-05-08T23:40:50.837769+0000","last_fullsized":"2024-05-08T23:40:50.837769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:52:57.586415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4528","reported_seq":4544,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.137874+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:41:12.137874+0000","last_peered":"2024-05-08T23:41:12.137874+0000","last_clean":"2024-05-08T23:41:12.137874+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:41:12.137874+0000","last_undegraded":"2024-05-08T23:41:12.137874+0000","last_fullsized":"2024-05-08T23:41:12.137874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":4528,"log_dups_size":0,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'6","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.715531+0000","last_change":"2024-05-08T23:32:09.893230+0000","last_active":"2024-05-08T23:40:50.715531+0000","last_peered":"2024-05-08T23:40:50.715531+0000","last_clean":"2024-05-08T23:40:50.715531+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:40:50.715531+0000","last_undegraded":"2024-05-08T23:40:50.715531+0000","last_fullsized":"2024-05-08T23:40:50.715531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:48:00.993056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7912","reported_seq":7928,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.153925+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:41:12.153925+0000","last_peered":"2024-05-08T23:41:12.153925+0000","last_clean":"2024-05-08T23:41:12.153925+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:41:12.153925+0000","last_undegraded":"2024-05-08T23:41:12.153925+0000","last_fullsized":"2024-05-08T23:41:12.153925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1096","reported_seq":1113,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.832597+0000","last_change":"2024-05-08T23:32:09.886190+0000","last_active":"2024-05-08T23:40:50.832597+0000","last_peered":"2024-05-08T23:40:50.832597+0000","last_clean":"2024-05-08T23:40:50.832597+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:40:50.832597+0000","last_undegraded":"2024-05-08T23:40:50.832597+0000","last_fullsized":"2024-05-08T23:40:50.832597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5451","reported_seq":5466,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.060043+0000","last_change":"2024-05-08T23:32:10.345496+0000","last_active":"2024-05-08T23:41:12.060043+0000","last_peered":"2024-05-08T23:41:12.060043+0000","last_clean":"2024-05-08T23:41:12.060043+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:41:12.060043+0000","last_undegraded":"2024-05-08T23:41:12.060043+0000","last_fullsized":"2024-05-08T23:41:12.060043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:19:26.674691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'27","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:11.853637+0000","last_change":"2024-05-08T23:32:09.887607+0000","last_active":"2024-05-08T23:41:11.853637+0000","last_peered":"2024-05-08T23:41:11.853637+0000","last_clean":"2024-05-08T23:41:11.853637+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:41:11.853637+0000","last_undegraded":"2024-05-08T23:41:11.853637+0000","last_fullsized":"2024-05-08T23:41:11.853637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:15:42.716876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5065","reported_seq":5081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.136265+0000","last_change":"2024-05-08T23:32:10.358891+0000","last_active":"2024-05-08T23:41:12.136265+0000","last_peered":"2024-05-08T23:41:12.136265+0000","last_clean":"2024-05-08T23:41:12.136265+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:41:12.136265+0000","last_undegraded":"2024-05-08T23:41:12.136265+0000","last_fullsized":"2024-05-08T23:41:12.136265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852477+0000","last_change":"2024-05-08T23:32:09.886219+0000","last_active":"2024-05-08T23:40:50.852477+0000","last_peered":"2024-05-08T23:40:50.852477+0000","last_clean":"2024-05-08T23:40:50.852477+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:40:50.852477+0000","last_undegraded":"2024-05-08T23:40:50.852477+0000","last_fullsized":"2024-05-08T23:40:50.852477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6601","reported_seq":6614,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.061346+0000","last_change":"2024-05-08T23:32:59.856036+0000","last_active":"2024-05-08T23:41:12.061346+0000","last_peered":"2024-05-08T23:41:12.061346+0000","last_clean":"2024-05-08T23:41:12.061346+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:41:12.061346+0000","last_undegraded":"2024-05-08T23:41:12.061346+0000","last_fullsized":"2024-05-08T23:41:12.061346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6601,"log_dups_size":0,"ondisk_log_size":6601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:07.549431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:39:33.542167+0000","last_change":"2024-05-08T23:32:09.879933+0000","last_active":"2024-05-08T23:39:33.542167+0000","last_peered":"2024-05-08T23:39:33.542167+0000","last_clean":"2024-05-08T23:39:33.542167+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:39:33.542167+0000","last_undegraded":"2024-05-08T23:39:33.542167+0000","last_fullsized":"2024-05-08T23:39:33.542167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7591","reported_seq":7607,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.067402+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:41:12.067402+0000","last_peered":"2024-05-08T23:41:12.067402+0000","last_clean":"2024-05-08T23:41:12.067402+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:41:12.067402+0000","last_undegraded":"2024-05-08T23:41:12.067402+0000","last_fullsized":"2024-05-08T23:41:12.067402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.812887+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:40:50.812887+0000","last_peered":"2024-05-08T23:40:50.812887+0000","last_clean":"2024-05-08T23:40:50.812887+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:40:50.812887+0000","last_undegraded":"2024-05-08T23:40:50.812887+0000","last_fullsized":"2024-05-08T23:40:50.812887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:05.375736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6503","reported_seq":6519,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.086682+0000","last_change":"2024-05-08T23:32:10.350060+0000","last_active":"2024-05-08T23:41:12.086682+0000","last_peered":"2024-05-08T23:41:12.086682+0000","last_clean":"2024-05-08T23:41:12.086682+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:41:12.086682+0000","last_undegraded":"2024-05-08T23:41:12.086682+0000","last_fullsized":"2024-05-08T23:41:12.086682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6503,"log_dups_size":0,"ondisk_log_size":6503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.852769+0000","last_change":"2024-05-08T23:32:09.888850+0000","last_active":"2024-05-08T23:40:50.852769+0000","last_peered":"2024-05-08T23:40:50.852769+0000","last_clean":"2024-05-08T23:40:50.852769+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:40:50.852769+0000","last_undegraded":"2024-05-08T23:40:50.852769+0000","last_fullsized":"2024-05-08T23:40:50.852769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5821","reported_seq":5845,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.061193+0000","last_change":"2024-05-08T23:32:59.859486+0000","last_active":"2024-05-08T23:41:12.061193+0000","last_peered":"2024-05-08T23:41:12.061193+0000","last_clean":"2024-05-08T23:41:12.061193+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:41:12.061193+0000","last_undegraded":"2024-05-08T23:41:12.061193+0000","last_fullsized":"2024-05-08T23:41:12.061193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'17","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.729049+0000","last_change":"2024-05-08T23:32:09.884486+0000","last_active":"2024-05-08T23:40:51.729049+0000","last_peered":"2024-05-08T23:40:51.729049+0000","last_clean":"2024-05-08T23:40:51.729049+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:40:51.729049+0000","last_undegraded":"2024-05-08T23:40:51.729049+0000","last_fullsized":"2024-05-08T23:40:51.729049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8452","reported_seq":8467,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.158534+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:41:12.158534+0000","last_peered":"2024-05-08T23:41:12.158534+0000","last_clean":"2024-05-08T23:41:12.158534+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:41:12.158534+0000","last_undegraded":"2024-05-08T23:41:12.158534+0000","last_fullsized":"2024-05-08T23:41:12.158534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8452,"log_dups_size":0,"ondisk_log_size":8452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:08.151522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'223","reported_seq":239,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.718908+0000","last_change":"2024-05-08T23:32:09.885879+0000","last_active":"2024-05-08T23:40:50.718908+0000","last_peered":"2024-05-08T23:40:50.718908+0000","last_clean":"2024-05-08T23:40:50.718908+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:40:50.718908+0000","last_undegraded":"2024-05-08T23:40:50.718908+0000","last_fullsized":"2024-05-08T23:40:50.718908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:43:00.140227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6156","reported_seq":6171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.138449+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:41:12.138449+0000","last_peered":"2024-05-08T23:41:12.138449+0000","last_clean":"2024-05-08T23:41:12.138449+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:41:12.138449+0000","last_undegraded":"2024-05-08T23:41:12.138449+0000","last_fullsized":"2024-05-08T23:41:12.138449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:23:54.106297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.870068+0000","last_change":"2024-05-08T23:32:09.887605+0000","last_active":"2024-05-08T23:40:50.870068+0000","last_peered":"2024-05-08T23:40:50.870068+0000","last_clean":"2024-05-08T23:40:50.870068+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:40:50.870068+0000","last_undegraded":"2024-05-08T23:40:50.870068+0000","last_fullsized":"2024-05-08T23:40:50.870068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5984","reported_seq":6000,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.078787+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:41:12.078787+0000","last_peered":"2024-05-08T23:41:12.078787+0000","last_clean":"2024-05-08T23:41:12.078787+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:41:12.078787+0000","last_undegraded":"2024-05-08T23:41:12.078787+0000","last_fullsized":"2024-05-08T23:41:12.078787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.901402+0000","last_change":"2024-05-08T23:32:09.881203+0000","last_active":"2024-05-08T23:40:50.901402+0000","last_peered":"2024-05-08T23:40:50.901402+0000","last_clean":"2024-05-08T23:40:50.901402+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:40:50.901402+0000","last_undegraded":"2024-05-08T23:40:50.901402+0000","last_fullsized":"2024-05-08T23:40:50.901402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7325","reported_seq":7340,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.167332+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:41:12.167332+0000","last_peered":"2024-05-08T23:41:12.167332+0000","last_clean":"2024-05-08T23:41:12.167332+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:41:12.167332+0000","last_undegraded":"2024-05-08T23:41:12.167332+0000","last_fullsized":"2024-05-08T23:41:12.167332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7325,"log_dups_size":0,"ondisk_log_size":7325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:57:16.493491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.831450+0000","last_change":"2024-05-08T23:32:09.885305+0000","last_active":"2024-05-08T23:40:50.831450+0000","last_peered":"2024-05-08T23:40:50.831450+0000","last_clean":"2024-05-08T23:40:50.831450+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:40:50.831450+0000","last_undegraded":"2024-05-08T23:40:50.831450+0000","last_fullsized":"2024-05-08T23:40:50.831450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:12:37.584292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7017","reported_seq":7032,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.069071+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:41:12.069071+0000","last_peered":"2024-05-08T23:41:12.069071+0000","last_clean":"2024-05-08T23:41:12.069071+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:41:12.069071+0000","last_undegraded":"2024-05-08T23:41:12.069071+0000","last_fullsized":"2024-05-08T23:41:12.069071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7017,"log_dups_size":0,"ondisk_log_size":7017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:40:08.328781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'15","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828797+0000","last_change":"2024-05-08T23:32:09.878543+0000","last_active":"2024-05-08T23:40:50.828797+0000","last_peered":"2024-05-08T23:40:50.828797+0000","last_clean":"2024-05-08T23:40:50.828797+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:40:50.828797+0000","last_undegraded":"2024-05-08T23:40:50.828797+0000","last_fullsized":"2024-05-08T23:40:50.828797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:35.029443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6611","reported_seq":6626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.012365+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:41:12.012365+0000","last_peered":"2024-05-08T23:41:12.012365+0000","last_clean":"2024-05-08T23:41:12.012365+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:41:12.012365+0000","last_undegraded":"2024-05-08T23:41:12.012365+0000","last_fullsized":"2024-05-08T23:41:12.012365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:59.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.836668+0000","last_change":"2024-05-08T23:32:09.893681+0000","last_active":"2024-05-08T23:40:50.836668+0000","last_peered":"2024-05-08T23:40:50.836668+0000","last_clean":"2024-05-08T23:40:50.836668+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:40:50.836668+0000","last_undegraded":"2024-05-08T23:40:50.836668+0000","last_fullsized":"2024-05-08T23:40:50.836668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:31:33.550915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6076","reported_seq":6100,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.038791+0000","last_change":"2024-05-08T23:33:00.026523+0000","last_active":"2024-05-08T23:41:12.038791+0000","last_peered":"2024-05-08T23:41:12.038791+0000","last_clean":"2024-05-08T23:41:12.038791+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:41:12.038791+0000","last_undegraded":"2024-05-08T23:41:12.038791+0000","last_fullsized":"2024-05-08T23:41:12.038791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:16:23.920007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.850771+0000","last_change":"2024-05-08T23:32:09.898720+0000","last_active":"2024-05-08T23:40:50.850771+0000","last_peered":"2024-05-08T23:40:50.850771+0000","last_clean":"2024-05-08T23:40:50.850771+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:40:50.850771+0000","last_undegraded":"2024-05-08T23:40:50.850771+0000","last_fullsized":"2024-05-08T23:40:50.850771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5813","reported_seq":5830,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.110338+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:41:12.110338+0000","last_peered":"2024-05-08T23:41:12.110338+0000","last_clean":"2024-05-08T23:41:12.110338+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:41:12.110338+0000","last_undegraded":"2024-05-08T23:41:12.110338+0000","last_fullsized":"2024-05-08T23:41:12.110338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:33.743927+0000","last_change":"2024-05-08T23:32:09.884201+0000","last_active":"2024-05-08T23:40:33.743927+0000","last_peered":"2024-05-08T23:40:33.743927+0000","last_clean":"2024-05-08T23:40:33.743927+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:40:33.743927+0000","last_undegraded":"2024-05-08T23:40:33.743927+0000","last_fullsized":"2024-05-08T23:40:33.743927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6336","reported_seq":6352,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.055508+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:41:12.055508+0000","last_peered":"2024-05-08T23:41:12.055508+0000","last_clean":"2024-05-08T23:41:12.055508+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:41:12.055508+0000","last_undegraded":"2024-05-08T23:41:12.055508+0000","last_fullsized":"2024-05-08T23:41:12.055508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:17.622154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6281","reported_seq":6298,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.063409+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:41:12.063409+0000","last_peered":"2024-05-08T23:41:12.063409+0000","last_clean":"2024-05-08T23:41:12.063409+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:41:12.063409+0000","last_undegraded":"2024-05-08T23:41:12.063409+0000","last_fullsized":"2024-05-08T23:41:12.063409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'27","reported_seq":68,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828876+0000","last_change":"2024-05-08T23:32:09.880419+0000","last_active":"2024-05-08T23:40:50.828876+0000","last_peered":"2024-05-08T23:40:50.828876+0000","last_clean":"2024-05-08T23:40:50.828876+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:40:50.828876+0000","last_undegraded":"2024-05-08T23:40:50.828876+0000","last_fullsized":"2024-05-08T23:40:50.828876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:43.307155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6855","reported_seq":6872,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.108299+0000","last_change":"2024-05-08T23:32:10.352761+0000","last_active":"2024-05-08T23:41:12.108299+0000","last_peered":"2024-05-08T23:41:12.108299+0000","last_clean":"2024-05-08T23:41:12.108299+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:41:12.108299+0000","last_undegraded":"2024-05-08T23:41:12.108299+0000","last_fullsized":"2024-05-08T23:41:12.108299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.815969+0000","last_change":"2024-05-08T23:32:09.887621+0000","last_active":"2024-05-08T23:40:50.815969+0000","last_peered":"2024-05-08T23:40:50.815969+0000","last_clean":"2024-05-08T23:40:50.815969+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:40:50.815969+0000","last_undegraded":"2024-05-08T23:40:50.815969+0000","last_fullsized":"2024-05-08T23:40:50.815969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:38:16.309946+0000","last_change":"2024-05-08T23:32:07.080155+0000","last_active":"2024-05-08T23:38:16.309946+0000","last_peered":"2024-05-08T23:38:16.309946+0000","last_clean":"2024-05-08T23:38:16.309946+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:38:16.309946+0000","last_undegraded":"2024-05-08T23:38:16.309946+0000","last_fullsized":"2024-05-08T23:38:16.309946+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:05.703491+0000","last_clean_scrub_stamp":"2024-05-08T23:32:05.703491+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:50.104873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6396","reported_seq":6413,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.072783+0000","last_change":"2024-05-08T23:32:10.358901+0000","last_active":"2024-05-08T23:41:12.072783+0000","last_peered":"2024-05-08T23:41:12.072783+0000","last_clean":"2024-05-08T23:41:12.072783+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:41:12.072783+0000","last_undegraded":"2024-05-08T23:41:12.072783+0000","last_fullsized":"2024-05-08T23:41:12.072783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6396,"log_dups_size":0,"ondisk_log_size":6396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.849138+0000","last_change":"2024-05-08T23:32:09.888429+0000","last_active":"2024-05-08T23:40:50.849138+0000","last_peered":"2024-05-08T23:40:50.849138+0000","last_clean":"2024-05-08T23:40:50.849138+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:40:50.849138+0000","last_undegraded":"2024-05-08T23:40:50.849138+0000","last_fullsized":"2024-05-08T23:40:50.849138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:12.289653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:38.734890+0000","last_change":"2024-05-08T23:32:09.895872+0000","last_active":"2024-05-08T23:40:38.734890+0000","last_peered":"2024-05-08T23:40:38.734890+0000","last_clean":"2024-05-08T23:40:38.734890+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:40:38.734890+0000","last_undegraded":"2024-05-08T23:40:38.734890+0000","last_fullsized":"2024-05-08T23:40:38.734890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:25.717619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7015","reported_seq":7032,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.034914+0000","last_change":"2024-05-08T23:32:10.344176+0000","last_active":"2024-05-08T23:41:12.034914+0000","last_peered":"2024-05-08T23:41:12.034914+0000","last_clean":"2024-05-08T23:41:12.034914+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:41:12.034914+0000","last_undegraded":"2024-05-08T23:41:12.034914+0000","last_fullsized":"2024-05-08T23:41:12.034914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'87","reported_seq":105,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:16.118969+0000","last_change":"2024-05-08T23:32:09.888911+0000","last_active":"2024-05-08T23:41:16.118969+0000","last_peered":"2024-05-08T23:41:16.118969+0000","last_clean":"2024-05-08T23:41:16.118969+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:41:16.118969+0000","last_undegraded":"2024-05-08T23:41:16.118969+0000","last_fullsized":"2024-05-08T23:41:16.118969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6431","reported_seq":6448,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051664+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:41:12.051664+0000","last_peered":"2024-05-08T23:41:12.051664+0000","last_clean":"2024-05-08T23:41:12.051664+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:41:12.051664+0000","last_undegraded":"2024-05-08T23:41:12.051664+0000","last_fullsized":"2024-05-08T23:41:12.051664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:51.423772+0000","last_change":"2024-05-08T23:32:09.888237+0000","last_active":"2024-05-08T23:40:51.423772+0000","last_peered":"2024-05-08T23:40:51.423772+0000","last_clean":"2024-05-08T23:40:51.423772+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:40:51.423772+0000","last_undegraded":"2024-05-08T23:40:51.423772+0000","last_fullsized":"2024-05-08T23:40:51.423772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:14.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5458","reported_seq":5475,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040763+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:41:12.040763+0000","last_peered":"2024-05-08T23:41:12.040763+0000","last_clean":"2024-05-08T23:41:12.040763+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:41:12.040763+0000","last_undegraded":"2024-05-08T23:41:12.040763+0000","last_fullsized":"2024-05-08T23:41:12.040763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.851008+0000","last_change":"2024-05-08T23:32:09.897229+0000","last_active":"2024-05-08T23:40:50.851008+0000","last_peered":"2024-05-08T23:40:50.851008+0000","last_clean":"2024-05-08T23:40:50.851008+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:40:50.851008+0000","last_undegraded":"2024-05-08T23:40:50.851008+0000","last_fullsized":"2024-05-08T23:40:50.851008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:38.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5844","reported_seq":5860,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:55.699542+0000","last_change":"2024-05-08T23:32:10.350739+0000","last_active":"2024-05-08T23:40:55.699542+0000","last_peered":"2024-05-08T23:40:55.699542+0000","last_clean":"2024-05-08T23:40:55.699542+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:40:55.699542+0000","last_undegraded":"2024-05-08T23:40:55.699542+0000","last_fullsized":"2024-05-08T23:40:55.699542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5844,"log_dups_size":0,"ondisk_log_size":5844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:14:35.535280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:57.095768+0000","last_change":"2024-05-08T23:32:09.886685+0000","last_active":"2024-05-08T23:40:57.095768+0000","last_peered":"2024-05-08T23:40:57.095768+0000","last_clean":"2024-05-08T23:40:57.095768+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:40:57.095768+0000","last_undegraded":"2024-05-08T23:40:57.095768+0000","last_fullsized":"2024-05-08T23:40:57.095768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6488","reported_seq":6505,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.041984+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:41:12.041984+0000","last_peered":"2024-05-08T23:41:12.041984+0000","last_clean":"2024-05-08T23:41:12.041984+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:41:12.041984+0000","last_undegraded":"2024-05-08T23:41:12.041984+0000","last_fullsized":"2024-05-08T23:41:12.041984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:02.096585+0000","last_change":"2024-05-08T23:32:09.891461+0000","last_active":"2024-05-08T23:41:02.096585+0000","last_peered":"2024-05-08T23:41:02.096585+0000","last_clean":"2024-05-08T23:41:02.096585+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:41:02.096585+0000","last_undegraded":"2024-05-08T23:41:02.096585+0000","last_fullsized":"2024-05-08T23:41:02.096585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5341","reported_seq":5358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.772081+0000","last_change":"2024-05-08T23:32:10.349433+0000","last_active":"2024-05-08T23:41:12.772081+0000","last_peered":"2024-05-08T23:41:12.772081+0000","last_clean":"2024-05-08T23:41:12.772081+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:41:12.772081+0000","last_undegraded":"2024-05-08T23:41:12.772081+0000","last_fullsized":"2024-05-08T23:41:12.772081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'685","reported_seq":702,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.883295+0000","last_change":"2024-05-08T23:32:09.891587+0000","last_active":"2024-05-08T23:40:50.883295+0000","last_peered":"2024-05-08T23:40:50.883295+0000","last_clean":"2024-05-08T23:40:50.883295+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:40:50.883295+0000","last_undegraded":"2024-05-08T23:40:50.883295+0000","last_fullsized":"2024-05-08T23:40:50.883295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":685,"log_dups_size":0,"ondisk_log_size":685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:04.978248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5938","reported_seq":5954,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051117+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:41:12.051117+0000","last_peered":"2024-05-08T23:41:12.051117+0000","last_clean":"2024-05-08T23:41:12.051117+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:41:12.051117+0000","last_undegraded":"2024-05-08T23:41:12.051117+0000","last_fullsized":"2024-05-08T23:41:12.051117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:03:03.705751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.896708+0000","last_change":"2024-05-08T23:32:09.885540+0000","last_active":"2024-05-08T23:40:50.896708+0000","last_peered":"2024-05-08T23:40:50.896708+0000","last_clean":"2024-05-08T23:40:50.896708+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:40:50.896708+0000","last_undegraded":"2024-05-08T23:40:50.896708+0000","last_fullsized":"2024-05-08T23:40:50.896708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:48.876610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6544","reported_seq":6560,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.821114+0000","last_change":"2024-05-08T23:32:10.354673+0000","last_active":"2024-05-08T23:41:12.821114+0000","last_peered":"2024-05-08T23:41:12.821114+0000","last_clean":"2024-05-08T23:41:12.821114+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:41:12.821114+0000","last_undegraded":"2024-05-08T23:41:12.821114+0000","last_fullsized":"2024-05-08T23:41:12.821114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:22.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'2","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:16.119067+0000","last_change":"2024-05-08T23:32:09.890493+0000","last_active":"2024-05-08T23:41:16.119067+0000","last_peered":"2024-05-08T23:41:16.119067+0000","last_clean":"2024-05-08T23:41:16.119067+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:41:16.119067+0000","last_undegraded":"2024-05-08T23:41:16.119067+0000","last_fullsized":"2024-05-08T23:41:16.119067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6824","reported_seq":6841,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:13.779035+0000","last_change":"2024-05-08T23:32:10.351206+0000","last_active":"2024-05-08T23:41:13.779035+0000","last_peered":"2024-05-08T23:41:13.779035+0000","last_clean":"2024-05-08T23:41:13.779035+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:41:13.779035+0000","last_undegraded":"2024-05-08T23:41:13.779035+0000","last_fullsized":"2024-05-08T23:41:13.779035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6824,"log_dups_size":0,"ondisk_log_size":6824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.828990+0000","last_change":"2024-05-08T23:32:09.881576+0000","last_active":"2024-05-08T23:40:50.828990+0000","last_peered":"2024-05-08T23:40:50.828990+0000","last_clean":"2024-05-08T23:40:50.828990+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:40:50.828990+0000","last_undegraded":"2024-05-08T23:40:50.828990+0000","last_fullsized":"2024-05-08T23:40:50.828990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:50.577947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7700","reported_seq":7715,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.162666+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:41:12.162666+0000","last_peered":"2024-05-08T23:41:12.162666+0000","last_clean":"2024-05-08T23:41:12.162666+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:41:12.162666+0000","last_undegraded":"2024-05-08T23:41:12.162666+0000","last_fullsized":"2024-05-08T23:41:12.162666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:10.433209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7806","reported_seq":7822,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.080477+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:41:12.080477+0000","last_peered":"2024-05-08T23:41:12.080477+0000","last_clean":"2024-05-08T23:41:12.080477+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:41:12.080477+0000","last_undegraded":"2024-05-08T23:41:12.080477+0000","last_fullsized":"2024-05-08T23:41:12.080477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7806,"log_dups_size":0,"ondisk_log_size":7806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:24:51.925820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6574","reported_seq":6590,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.089637+0000","last_change":"2024-05-08T23:32:10.345618+0000","last_active":"2024-05-08T23:41:12.089637+0000","last_peered":"2024-05-08T23:41:12.089637+0000","last_clean":"2024-05-08T23:41:12.089637+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:41:12.089637+0000","last_undegraded":"2024-05-08T23:41:12.089637+0000","last_fullsized":"2024-05-08T23:41:12.089637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:34:46.453733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8085","reported_seq":8102,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.170588+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:41:12.170588+0000","last_peered":"2024-05-08T23:41:12.170588+0000","last_clean":"2024-05-08T23:41:12.170588+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:41:12.170588+0000","last_undegraded":"2024-05-08T23:41:12.170588+0000","last_fullsized":"2024-05-08T23:41:12.170588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5657","reported_seq":5674,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.097092+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:41:12.097092+0000","last_peered":"2024-05-08T23:41:12.097092+0000","last_clean":"2024-05-08T23:41:12.097092+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:41:12.097092+0000","last_undegraded":"2024-05-08T23:41:12.097092+0000","last_fullsized":"2024-05-08T23:41:12.097092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6017","reported_seq":6034,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.073073+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:41:12.073073+0000","last_peered":"2024-05-08T23:41:12.073073+0000","last_clean":"2024-05-08T23:41:12.073073+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:41:12.073073+0000","last_undegraded":"2024-05-08T23:41:12.073073+0000","last_fullsized":"2024-05-08T23:41:12.073073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7053","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.066767+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:41:12.066767+0000","last_peered":"2024-05-08T23:41:12.066767+0000","last_clean":"2024-05-08T23:41:12.066767+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:41:12.066767+0000","last_undegraded":"2024-05-08T23:41:12.066767+0000","last_fullsized":"2024-05-08T23:41:12.066767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5666","reported_seq":5682,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.152832+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:41:12.152832+0000","last_peered":"2024-05-08T23:41:12.152832+0000","last_clean":"2024-05-08T23:41:12.152832+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:41:12.152832+0000","last_undegraded":"2024-05-08T23:41:12.152832+0000","last_fullsized":"2024-05-08T23:41:12.152832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:23.632138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7698","reported_seq":7715,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:11.975204+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:41:11.975204+0000","last_peered":"2024-05-08T23:41:11.975204+0000","last_clean":"2024-05-08T23:41:11.975204+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:41:11.975204+0000","last_undegraded":"2024-05-08T23:41:11.975204+0000","last_fullsized":"2024-05-08T23:41:11.975204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5557","reported_seq":5574,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.100041+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:41:12.100041+0000","last_peered":"2024-05-08T23:41:12.100041+0000","last_clean":"2024-05-08T23:41:12.100041+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:41:12.100041+0000","last_undegraded":"2024-05-08T23:41:12.100041+0000","last_fullsized":"2024-05-08T23:41:12.100041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5557,"log_dups_size":0,"ondisk_log_size":5557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6393","reported_seq":6410,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.772040+0000","last_change":"2024-05-08T23:32:10.357140+0000","last_active":"2024-05-08T23:41:12.772040+0000","last_peered":"2024-05-08T23:41:12.772040+0000","last_clean":"2024-05-08T23:41:12.772040+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:41:12.772040+0000","last_undegraded":"2024-05-08T23:41:12.772040+0000","last_fullsized":"2024-05-08T23:41:12.772040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6393,"log_dups_size":0,"ondisk_log_size":6393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7241","reported_seq":7258,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:13.778962+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:41:13.778962+0000","last_peered":"2024-05-08T23:41:13.778962+0000","last_clean":"2024-05-08T23:41:13.778962+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:41:13.778962+0000","last_undegraded":"2024-05-08T23:41:13.778962+0000","last_fullsized":"2024-05-08T23:41:13.778962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6810","reported_seq":6827,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:15.549263+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:41:15.549263+0000","last_peered":"2024-05-08T23:41:15.549263+0000","last_clean":"2024-05-08T23:41:15.549263+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:41:15.549263+0000","last_undegraded":"2024-05-08T23:41:15.549263+0000","last_fullsized":"2024-05-08T23:41:15.549263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6394","reported_seq":6410,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.083733+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:41:12.083733+0000","last_peered":"2024-05-08T23:41:12.083733+0000","last_clean":"2024-05-08T23:41:12.083733+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:41:12.083733+0000","last_undegraded":"2024-05-08T23:41:12.083733+0000","last_fullsized":"2024-05-08T23:41:12.083733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:31.515403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419416,"ondisk_log_size":419416,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293737,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2683,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34587474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2654,"ondisk_log_size":2654,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633788,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":620670,"kb_avail":93738052,"statfs":{"total":96636764160,"available":95987765248,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":635566539},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":11.551,"5min":8.4469999999999992,"15min":3.2210000000000001},"min":{"1min":0.28199999999999997,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":45.125999999999998,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.623},{"interface":"front","average":{"1min":9.0259999999999998,"5min":7.827,"15min":3.0419999999999998},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":37.293999999999997,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.34699999999999998}]},{"osd":1,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.194,"5min":4.4950000000000001,"15min":1.9470000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":13.531000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":9.1940000000000008,"5min":7.9210000000000003,"15min":3.1160000000000001},"min":{"1min":0.40000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":24.387,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":4.5529999999999999,"5min":4.2610000000000001,"15min":1.8879999999999999},"min":{"1min":0.51300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":15.938000000000001,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.44},{"interface":"front","average":{"1min":6.0819999999999999,"5min":5.0229999999999997,"15min":2.1219999999999999},"min":{"1min":0.41999999999999998,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":19.565000000000001,"5min":26.084,"15min":26.084},"last":0.38300000000000001}]},{"osd":3,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":6.0049999999999999,"5min":7.1989999999999998,"15min":2.8839999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":19.614000000000001,"5min":52.869,"15min":52.869},"last":0.59899999999999998},{"interface":"front","average":{"1min":10.468,"5min":7.2720000000000002,"15min":2.855},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.35499999999999998},"max":{"1min":49.421999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.504}]},{"osd":4,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":5.0599999999999996,"15min":2.0430000000000001},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.9139999999999997,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":5.3949999999999996,"5min":3.8919999999999999,"15min":1.6950000000000001},"min":{"1min":0.39300000000000002,"5min":0.37,"15min":0.37},"max":{"1min":22.901,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":7.1719999999999997,"5min":5.1349999999999998,"15min":2.0910000000000002},"min":{"1min":0.502,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":18.988,"5min":63.732999999999997,"15min":63.732999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":9.2859999999999996,"5min":7.7160000000000002,"15min":2.9860000000000002},"min":{"1min":0.69199999999999995,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":18.504999999999999,"5min":62.558,"15min":62.558},"last":0.73199999999999998}]},{"osd":6,"last update":"Wed May 8 23:40:23 2024","interfaces":[{"interface":"back","average":{"1min":10.367000000000001,"5min":7.5259999999999998,"15min":2.9529999999999998},"min":{"1min":0.71799999999999997,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":45.826999999999998,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":5.8150000000000004,"5min":4.6600000000000001,"15min":1.982},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40600000000000003},"max":{"1min":38.801000000000002,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.58699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574960,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4710076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.7320000000000002,"5min":6.8529999999999998,"15min":2.7749999999999999},"min":{"1min":0.52500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":19.369,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":6.4900000000000002,"5min":7.8170000000000002,"15min":3.1259999999999999},"min":{"1min":0.47199999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":18.379000000000001,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.51900000000000002}]},{"osd":1,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":6.8890000000000002,"5min":6.3970000000000002,"15min":2.8570000000000002},"min":{"1min":0.876,"5min":0.313,"15min":0.313},"max":{"1min":24.693000000000001,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":5.6459999999999999,"5min":5.7210000000000001,"15min":2.6230000000000002},"min":{"1min":0.57899999999999996,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":24.187999999999999,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.44600000000000001}]},{"osd":2,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":4.306,"5min":4.7060000000000004,"15min":2.1030000000000002},"min":{"1min":0.58499999999999996,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":15.589,"5min":41.445,"15min":41.445},"last":0.58399999999999996},{"interface":"front","average":{"1min":4.3399999999999999,"5min":4.6550000000000002,"15min":2.0630000000000002},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":15.624000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.66400000000000003}]},{"osd":3,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":5.4909999999999997,"5min":5.2729999999999997,"15min":2.266},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":15.467000000000001,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":5.8230000000000004,"5min":7.4550000000000001,"15min":3.1749999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":28.600000000000001,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":3.141,"15min":1.446},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":6.891,"5min":20.913,"15min":20.913},"last":0.62},{"interface":"front","average":{"1min":2.1909999999999998,"5min":3.875,"15min":1.8129999999999999},"min":{"1min":0.52700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":6.9569999999999999,"5min":35.975999999999999,"15min":35.975999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":7.3280000000000003,"5min":6.5369999999999999,"15min":2.657},"min":{"1min":0.64500000000000002,"5min":0.39200000000000002,"15min":0.33100000000000002},"max":{"1min":36.756,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":6.0759999999999996,"5min":4.633,"15min":2.0230000000000001},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":23.105,"5min":23.105,"15min":23.105},"last":0.56799999999999995}]},{"osd":7,"last update":"Wed May 8 23:40:18 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":4.2359999999999998,"15min":2.0259999999999998},"min":{"1min":0.44800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":6.9900000000000002,"5min":24.16,"15min":33.463000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":5.7530000000000001,"5min":5.6429999999999998,"15min":2.3460000000000001},"min":{"1min":0.83299999999999996,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.056999999999999,"5min":45.805,"15min":45.805},"last":0.54100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":7.673,"5min":6.6539999999999999,"15min":2.5219999999999998},"min":{"1min":0.437,"5min":0.29099999999999998,"15min":0.23400000000000001},"max":{"1min":30.721,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":4.6159999999999997,"5min":5.1420000000000003,"15min":2.1499999999999999},"min":{"1min":0.28299999999999997,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":13.93,"5min":48.844999999999999,"15min":48.844999999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.6609999999999996,"5min":4.5510000000000002,"15min":1.897},"min":{"1min":0.32400000000000001,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":22.721,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":6.5650000000000004,"5min":5.5819999999999999,"15min":2.3359999999999999},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":25.683,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.3979999999999997,"5min":7.8360000000000003,"15min":2.9780000000000002},"min":{"1min":0.28799999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":21.975000000000001,"5min":52.712000000000003,"15min":52.712000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":4.9900000000000002,"5min":9.1709999999999994,"15min":3.4780000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.32100000000000001},"max":{"1min":25.724,"5min":61.832000000000001,"15min":61.832000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":6.2939999999999996,"5min":6.4749999999999996,"15min":2.6850000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":23.529,"5min":45.267000000000003,"15min":45.267000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":7.2350000000000003,"5min":6.3070000000000004,"15min":2.5979999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":31.245999999999999,"5min":43.353999999999999,"15min":43.353999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":5.3419999999999996,"5min":6.3719999999999999,"15min":2.6850000000000001},"min":{"1min":0.66300000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":16.907,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.628},{"interface":"front","average":{"1min":6.984,"5min":8.1400000000000006,"15min":3.2719999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":36.280999999999999,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":4.4470000000000001,"5min":7.5030000000000001,"15min":3.0939999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":11.292999999999999,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":6.8490000000000002,"5min":7.016,"15min":2.8900000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":25.600999999999999,"5min":42.603999999999999,"15min":42.603999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed May 8 23:40:27 2024","interfaces":[{"interface":"back","average":{"1min":10.384,"5min":8.8420000000000005,"15min":3.4609999999999999},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":72.088999999999999,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":10.420999999999999,"5min":9.4009999999999998,"15min":3.7349999999999999},"min":{"1min":0.49399999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":72.150000000000006,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.324,"5min":6.6020000000000003,"15min":2.8780000000000001},"min":{"1min":0.28599999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":47.241999999999997,"5min":47.241999999999997,"15min":101.899},"last":0.35999999999999999},{"interface":"front","average":{"1min":7.3029999999999999,"5min":5.3879999999999999,"15min":2.5339999999999998},"min":{"1min":0.65000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":22.170999999999999,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.40899999999999997}]},{"osd":2,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":14.221,"5min":8.1920000000000002,"15min":3.5609999999999999},"min":{"1min":0.33900000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":134.16300000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":10.577999999999999,"5min":7.7560000000000002,"15min":3.3380000000000001},"min":{"1min":0.39800000000000002,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":47.277000000000001,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.496}]},{"osd":3,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.8740000000000006,"5min":7.0720000000000001,"15min":3.1829999999999998},"min":{"1min":0.59399999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":59.796999999999997,"5min":59.796999999999997,"15min":101.929},"last":0.47999999999999998},{"interface":"front","average":{"1min":4.3179999999999996,"5min":4.8780000000000001,"15min":2.2130000000000001},"min":{"1min":0.47499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":12.606,"5min":95.200999999999993,"15min":95.200999999999993},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":12.207000000000001,"5min":6.8620000000000001,"15min":3.206},"min":{"1min":0.79500000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":47.194000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.67900000000000005},{"interface":"front","average":{"1min":4.2290000000000001,"5min":5.282,"15min":2.3500000000000001},"min":{"1min":0.54300000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":12.694000000000001,"5min":39.813000000000002,"15min":44.545999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.326000000000001,"5min":6.3300000000000001,"15min":2.8500000000000001},"min":{"1min":1.111,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":38.494,"5min":38.494,"15min":78.780000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":5.0940000000000003,"5min":6.5419999999999998,"15min":2.7690000000000001},"min":{"1min":0.79800000000000004,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":12.657999999999999,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":8.0920000000000005,"5min":7.1189999999999998,"15min":3.2959999999999998},"min":{"1min":0.85299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":45.744999999999997,"5min":74.013000000000005,"15min":101.869},"last":0.76100000000000001},{"interface":"front","average":{"1min":10.259,"5min":6.2560000000000002,"15min":2.903},"min":{"1min":0.46100000000000002,"5min":0.317,"15min":0.317},"max":{"1min":38.545000000000002,"5min":71.323999999999998,"15min":78.647999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Wed May 8 23:40:24 2024","interfaces":[{"interface":"back","average":{"1min":11.394,"5min":6,"15min":2.8479999999999999},"min":{"1min":0.67200000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":37.725000000000001,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.75},{"interface":"front","average":{"1min":4.5819999999999999,"5min":4.9059999999999997,"15min":2.2759999999999998},"min":{"1min":0.61199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":13.746,"5min":27.056000000000001,"15min":44.713999999999999},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":5.266,"15min":2.2570000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":9.6880000000000006,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.3129999999999999,"5min":4.6859999999999999,"15min":2.149},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.27700000000000002},"max":{"1min":9.7699999999999996,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":5.0739999999999998,"15min":2.5710000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":1.9159999999999999,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.38},{"interface":"front","average":{"1min":1.2330000000000001,"5min":5.4859999999999998,"15min":2.524},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.318},"max":{"1min":7.0469999999999997,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.33900000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":7.5650000000000004,"15min":3.516},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27600000000000002},"max":{"1min":22.989999999999998,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.622,"5min":4.0359999999999996,"15min":2.0750000000000002},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":1.6659999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.1959999999999997,"15min":2.363},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":29.513999999999999,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":3.8719999999999999,"15min":2.0150000000000001},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":7.0970000000000004,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.499}]},{"osd":5,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":5.6399999999999997,"15min":2.8500000000000001},"min":{"1min":0.49199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":12.858000000000001,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.853,"5min":5.0960000000000001,"15min":2.371},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":12.723000000000001,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":4.8440000000000003,"15min":2.278},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":10.128,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.647,"5min":5.8680000000000003,"15min":2.605},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":9.3510000000000009,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":4.9199999999999999,"15min":2.3759999999999999},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.5679999999999996,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":2.2879999999999998,"5min":6.5650000000000004,"15min":2.96},"min":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.9250000000000007,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.54100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":4.125,"5min":6.7089999999999996,"15min":2.5619999999999998},"min":{"1min":0.35299999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":18.997,"5min":67.305999999999997,"15min":67.305999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":6.2910000000000004,"5min":9.1940000000000008,"15min":3.3969999999999998},"min":{"1min":0.32400000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":22.149999999999999,"5min":106.934,"15min":106.934},"last":0.34000000000000002}]},{"osd":1,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":9.6839999999999993,"5min":7.0439999999999996,"15min":2.6669999999999998},"min":{"1min":0.40699999999999997,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":76.628,"5min":76.628,"15min":76.628},"last":0.36199999999999999},{"interface":"front","average":{"1min":4.9690000000000003,"5min":6.4690000000000003,"15min":2.5129999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":19.045999999999999,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":8.5600000000000005,"5min":7.0899999999999999,"15min":3.008},"min":{"1min":0.371,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":57.378,"5min":57.378,"15min":57.378},"last":0.46999999999999997},{"interface":"front","average":{"1min":4.3460000000000001,"5min":5.8959999999999999,"15min":2.363},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":19.800000000000001,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.51400000000000001}]},{"osd":4,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":11.138999999999999,"5min":8.3179999999999996,"15min":3.2629999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":57.313000000000002,"5min":57.313000000000002,"15min":57.313000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":11.608000000000001,"5min":7.4400000000000004,"15min":2.9350000000000001},"min":{"1min":1.5469999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":53.343000000000004,"5min":68.238,"15min":68.238},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.965,"5min":6.7839999999999998,"15min":2.722},"min":{"1min":0.60999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":53.381,"5min":53.381,"15min":53.381},"last":0.68000000000000005},{"interface":"front","average":{"1min":6.71,"5min":7.867,"15min":3.1309999999999998},"min":{"1min":0.47099999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":19.084,"5min":36.213999999999999,"15min":36.213999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":10.84,"5min":7.8650000000000002,"15min":3.0960000000000001},"min":{"1min":1.2270000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":57.411000000000001,"5min":57.411000000000001,"15min":57.411000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":6.2850000000000001,"5min":5.6790000000000003,"15min":2.3849999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":19.113,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.624}]},{"osd":7,"last update":"Wed May 8 23:40:17 2024","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":9.2200000000000006,"15min":3.5830000000000002},"min":{"1min":0.59399999999999997,"5min":0.375,"15min":0.375},"max":{"1min":63.896999999999998,"5min":63.896999999999998,"15min":63.896999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":9.6880000000000006,"5min":7.1580000000000004,"15min":2.9129999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":53.182000000000002,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":6.1509999999999998,"15min":2.8980000000000001},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.3620000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":2.9729999999999999,"5min":6.7389999999999999,"15min":3.109},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":10.945,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.67200000000000004}]},{"osd":1,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7050000000000001,"5min":5.7030000000000003,"15min":2.9729999999999999},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":24.324999999999999,"5min":35.807000000000002,"15min":40.901000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":5.6130000000000004,"5min":5.867,"15min":2.8839999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":47.481999999999999,"5min":47.481999999999999,"15min":47.481999999999999},"last":0.54000000000000004}]},{"osd":2,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.9279999999999999,"5min":5.4900000000000002,"15min":2.871},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":13.356999999999999,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":2.7730000000000001,"5min":4.8339999999999996,"15min":2.5529999999999999},"min":{"1min":0.42199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":11.509,"5min":25.616,"15min":58.899000000000001},"last":0.499}]},{"osd":3,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.835,"15min":2.859},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.318},"max":{"1min":24.460999999999999,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":4.3490000000000002,"5min":7.7370000000000001,"15min":3.4350000000000001},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":31.451000000000001,"5min":111.221,"15min":111.221},"last":0.69999999999999996}]},{"osd":5,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":4.6660000000000004,"15min":2.266},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.7229999999999999,"5min":43.497999999999998,"15min":51.871000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":1.8460000000000001,"5min":4.4720000000000004,"15min":2.0409999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.7100000000000009,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2930000000000001,"5min":4.5890000000000004,"15min":2.0680000000000001},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.436999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":2.2069999999999999,"5min":4.548,"15min":2.3849999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":11.083,"5min":43.774999999999999,"15min":51.932000000000002},"last":0.628}]},{"osd":7,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":4.0289999999999999,"15min":1.9630000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":7.4790000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":2.6930000000000001,"5min":4.7949999999999999,"15min":2.4729999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":14.221,"5min":44.783000000000001,"15min":51.978000000000002},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":5.4900000000000002,"15min":2.7029999999999998},"min":{"1min":0.28599999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.4160000000000004,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.436},{"interface":"front","average":{"1min":1.8300000000000001,"5min":5.2009999999999996,"15min":2.3119999999999998},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.30199999999999999},"max":{"1min":7.7690000000000001,"5min":44.439,"15min":44.439},"last":0.498}]},{"osd":1,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":4.9509999999999996,"15min":2.9039999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":7.2759999999999998,"5min":31.355,"15min":124.85599999999999},"last":0.377},{"interface":"front","average":{"1min":1.347,"5min":4.8479999999999999,"15min":2.4580000000000002},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.317},"max":{"1min":7.327,"5min":31.385000000000002,"15min":41.060000000000002},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":6.5110000000000001,"15min":2.9790000000000001},"min":{"1min":0.45600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.8650000000000002,"5min":44.247,"15min":44.247},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.7170000000000001,"5min":4.8680000000000003,"15min":2.3039999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":7.5350000000000001,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":5.0279999999999996,"5min":7.7690000000000001,"15min":3.1699999999999999},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":39.265000000000001,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":4.1520000000000001,"5min":7.9029999999999996,"15min":4.0640000000000001},"min":{"1min":0.46100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.193000000000001,"5min":43.343000000000004,"15min":49.991999999999997},"last":0.47099999999999997}]},{"osd":4,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.1840000000000002,"15min":2.181},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":8.8119999999999994,"5min":43.305,"15min":43.305},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.4199999999999999,"5min":3.669,"15min":1.8049999999999999},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.3780000000000001,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.51800000000000002}]},{"osd":6,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":5.4550000000000001,"15min":2.855},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":7.8280000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":2.0369999999999999,"5min":4.7329999999999997,"15min":2.306},"min":{"1min":0.45000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.2289999999999992,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":5.6580000000000004,"15min":2.532},"min":{"1min":0.44500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":15.589,"5min":43.262,"15min":43.262},"last":0.53000000000000003},{"interface":"front","average":{"1min":2.2690000000000001,"5min":6.3700000000000001,"15min":3.3159999999999998},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":10.456,"5min":44.018000000000001,"15min":80.302000000000007},"last":0.65000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4622048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.972 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.973 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.974 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.975 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.976 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.977 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.978 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T23:32:05.703491+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=5, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.979 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.980 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.981 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.982 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:19.983 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T23:41:21.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:41:21.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:41:39.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:41:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.135+0000 7fe76d034700 1 Processor -- start 2024-05-08T23:41:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.135+0000 7fe76d034700 1 -- start start 2024-05-08T23:41:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76d034700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fe768152350 0x7fe768158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76d034700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe7680c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76d034700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe768070450 con 0x7fe7680c9a60 2024-05-08T23:41:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76d034700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7680c2e90 con 0x7fe768152350 2024-05-08T23:41:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76d034700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe768070730 con 0x7fe7680c38a0 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76759e700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe7680c9a60 0x7fe7680c85a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59186/0 (socket says 172.21.15.33:59186) 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76759e700 1 -- 172.21.15.33:0/1825916226 learned_addr learned my addr 172.21.15.33:0/1825916226 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1825916226 >> v2:172.21.15.42:3300/0 conn(0x7fe768152350 0x7fe768158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe76659c700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe7680c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128364456 0 0) 0x7fe768070450 con 0x7fe7680c9a60 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.136+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7540035b0 con 0x7fe7680c9a60 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe766d9d700 1 -- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 msgr2=0x7fe7680c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:41:40.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147285673 0 0) 0x7fe7540035b0 con 0x7fe7680c9a60 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe7680c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe766d9d700 1 -- 172.21.15.33:0/1825916226 >> v1:172.21.15.33:6789/0 conn(0x7fe7680c9a60 legacy=0x7fe7680c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe766d9d700 1 -- 172.21.15.33:0/1825916226 --> v2:172.21.15.42:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768070a10 con 0x7fe768152350 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe76659c700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe7680c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe768152350 0x7fe768158110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe758009840 tx=0x7fe758009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1721e3aa874c682a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 <== mon.1 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600019f0 con 0x7fe768152350 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 <== mon.1 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe758004030 con 0x7fe768152350 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.137+0000 7fe765d9b700 1 -- 172.21.15.33:0/1825916226 <== mon.1 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7580043c0 con 0x7fe768152350 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 -- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe768152350 msgr2=0x7fe768158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe768152350 0x7fe768158110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe758009840 tx=0x7fe758009b50 comp rx=0 tx=0).stop 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 -- 172.21.15.33:0/1825916226 shutdown_connections 2024-05-08T23:41:40.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe768152350 0x7fe768158110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 --2- 172.21.15.33:0/1825916226 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe7680c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 -- 172.21.15.33:0/1825916226 >> 172.21.15.33:0/1825916226 conn(0x7fe7680666c0 msgr2=0x7fe768066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 -- 172.21.15.33:0/1825916226 shutdown_connections 2024-05-08T23:41:40.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.138+0000 7fe76d034700 1 -- 172.21.15.33:0/1825916226 wait complete. 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.139+0000 7fe76d034700 1 Processor -- start 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 -- start start 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe768157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 0x7fe768152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681ee510 con 0x7fe768152350 2024-05-08T23:41:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681ee930 con 0x7fe7680c9a60 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.140+0000 7fe76d034700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe768153880 con 0x7fe7680c38a0 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 0x7fe768152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46106/0 (socket says 172.21.15.33:46106) 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 -- 172.21.15.33:0/1533812650 learned_addr learned my addr 172.21.15.33:0/1533812650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe768157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 -- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 msgr2=0x7fe768157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe768157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 -- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 msgr2=0x7fe768152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 0x7fe768152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 -- 172.21.15.33:0/1533812650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500094a0 con 0x7fe768152350 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76659c700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 0x7fe768152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76759e700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe760001ba0 tx=0x7fe76000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000aac0 con 0x7fe768152350 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000ac20 con 0x7fe768152350 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7580094a0 con 0x7fe768152350 2024-05-08T23:41:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600070d0 con 0x7fe768152350 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.141+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7681edb70 con 0x7fe768152350 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.142+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7681ee650 con 0x7fe768152350 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.143+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fe760007230 con 0x7fe768152350 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.143+0000 7fe75f7fe700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 0x7fe74806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.143+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fe760088f80 con 0x7fe768152350 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.143+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 0x7fe74806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:41:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.144+0000 7fe766d9d700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 0x7fe74806d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe7580090b0 tx=0x7fe758008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:41:40.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.148+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fe7600bc900 con 0x7fe768152350 2024-05-08T23:41:40.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.361+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe768070110 con 0x7fe74806ac90 2024-05-08T23:41:40.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.369+0000 7fe75f7fe700 1 -- 172.21.15.33:0/1533812650 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289474 (secure 0 0 0) 0x7fe7580042a0 con 0x7fe74806ac90 2024-05-08T23:41:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:41:40.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 msgr2=0x7fe74806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 0x7fe74806d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe7580090b0 tx=0x7fe758008030 comp rx=0 tx=0).stop 2024-05-08T23:41:40.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 msgr2=0x7fe7681478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe760001ba0 tx=0x7fe76000cf50 comp rx=0 tx=0).stop 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 shutdown_connections 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fe74806ac90 0x7fe74806d150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe768152350 0x7fe7681478c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fe7680c38a0 0x7fe768157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 --2- 172.21.15.33:0/1533812650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7680c9a60 0x7fe768152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 >> 172.21.15.33:0/1533812650 conn(0x7fe7680666c0 msgr2=0x7fe7680c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 shutdown_connections 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:41:40.373+0000 7fe76d034700 1 -- 172.21.15.33:0/1533812650 wait complete. 2024-05-08T23:41:40.373 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:41:40.384 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-05-08T23:41:39.502190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17884105,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320951,"num_write_kb":49019247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422212,"ondisk_log_size":422212,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4048792,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4011827,"kb_avail":750925928,"statfs":{"total":773094113280,"available":768948150272,"internally_reserved":0,"allocated":37773312,"data_stored":36472434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4108111416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003680"},"pg_stats":[{"pgid":"3.3f","version":"23'5702","reported_seq":5719,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.150598+0000","last_change":"2024-05-08T23:32:10.360338+0000","last_active":"2024-05-08T23:41:12.150598+0000","last_peered":"2024-05-08T23:41:12.150598+0000","last_clean":"2024-05-08T23:41:12.150598+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:41:12.150598+0000","last_undegraded":"2024-05-08T23:41:12.150598+0000","last_fullsized":"2024-05-08T23:41:12.150598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8392","reported_seq":8409,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040666+0000","last_change":"2024-05-08T23:32:10.356359+0000","last_active":"2024-05-08T23:41:12.040666+0000","last_peered":"2024-05-08T23:41:12.040666+0000","last_clean":"2024-05-08T23:41:12.040666+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:41:12.040666+0000","last_undegraded":"2024-05-08T23:41:12.040666+0000","last_fullsized":"2024-05-08T23:41:12.040666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6514","reported_seq":6531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.038998+0000","last_change":"2024-05-08T23:32:10.350469+0000","last_active":"2024-05-08T23:41:12.038998+0000","last_peered":"2024-05-08T23:41:12.038998+0000","last_clean":"2024-05-08T23:41:12.038998+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:41:12.038998+0000","last_undegraded":"2024-05-08T23:41:12.038998+0000","last_fullsized":"2024-05-08T23:41:12.038998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5995","reported_seq":6012,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.086784+0000","last_change":"2024-05-08T23:32:10.351472+0000","last_active":"2024-05-08T23:41:12.086784+0000","last_peered":"2024-05-08T23:41:12.086784+0000","last_clean":"2024-05-08T23:41:12.086784+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:41:12.086784+0000","last_undegraded":"2024-05-08T23:41:12.086784+0000","last_fullsized":"2024-05-08T23:41:12.086784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7607","reported_seq":7621,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.030041+0000","last_change":"2024-05-08T23:32:59.855454+0000","last_active":"2024-05-08T23:41:12.030041+0000","last_peered":"2024-05-08T23:41:12.030041+0000","last_clean":"2024-05-08T23:41:12.030041+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:41:12.030041+0000","last_undegraded":"2024-05-08T23:41:12.030041+0000","last_fullsized":"2024-05-08T23:41:12.030041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5753","reported_seq":5769,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.172968+0000","last_change":"2024-05-08T23:32:10.356380+0000","last_active":"2024-05-08T23:41:12.172968+0000","last_peered":"2024-05-08T23:41:12.172968+0000","last_clean":"2024-05-08T23:41:12.172968+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:41:12.172968+0000","last_undegraded":"2024-05-08T23:41:12.172968+0000","last_fullsized":"2024-05-08T23:41:12.172968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5753,"log_dups_size":0,"ondisk_log_size":5753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6497","reported_seq":6513,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051409+0000","last_change":"2024-05-08T23:32:10.353033+0000","last_active":"2024-05-08T23:41:12.051409+0000","last_peered":"2024-05-08T23:41:12.051409+0000","last_clean":"2024-05-08T23:41:12.051409+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:41:12.051409+0000","last_undegraded":"2024-05-08T23:41:12.051409+0000","last_fullsized":"2024-05-08T23:41:12.051409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6893","reported_seq":6909,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.108812+0000","last_change":"2024-05-08T23:32:10.348270+0000","last_active":"2024-05-08T23:41:12.108812+0000","last_peered":"2024-05-08T23:41:12.108812+0000","last_clean":"2024-05-08T23:41:12.108812+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:41:12.108812+0000","last_undegraded":"2024-05-08T23:41:12.108812+0000","last_fullsized":"2024-05-08T23:41:12.108812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8092","reported_seq":8108,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.163927+0000","last_change":"2024-05-08T23:32:10.347647+0000","last_active":"2024-05-08T23:41:12.163927+0000","last_peered":"2024-05-08T23:41:12.163927+0000","last_clean":"2024-05-08T23:41:12.163927+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:41:12.163927+0000","last_undegraded":"2024-05-08T23:41:12.163927+0000","last_fullsized":"2024-05-08T23:41:12.163927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8092,"log_dups_size":0,"ondisk_log_size":8092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7426","reported_seq":7442,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.159087+0000","last_change":"2024-05-08T23:32:10.355631+0000","last_active":"2024-05-08T23:41:12.159087+0000","last_peered":"2024-05-08T23:41:12.159087+0000","last_clean":"2024-05-08T23:41:12.159087+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:41:12.159087+0000","last_undegraded":"2024-05-08T23:41:12.159087+0000","last_fullsized":"2024-05-08T23:41:12.159087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7426,"log_dups_size":0,"ondisk_log_size":7426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6322","reported_seq":6338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.046460+0000","last_change":"2024-05-08T23:32:10.354969+0000","last_active":"2024-05-08T23:41:12.046460+0000","last_peered":"2024-05-08T23:41:12.046460+0000","last_clean":"2024-05-08T23:41:12.046460+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:41:12.046460+0000","last_undegraded":"2024-05-08T23:41:12.046460+0000","last_fullsized":"2024-05-08T23:41:12.046460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5509","reported_seq":5536,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.529076+0000","last_change":"2024-05-08T23:41:29.529218+0000","last_active":"2024-05-08T23:41:29.529076+0000","last_peered":"2024-05-08T23:41:29.529076+0000","last_clean":"2024-05-08T23:41:29.529076+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:41:29.529076+0000","last_undegraded":"2024-05-08T23:41:29.529076+0000","last_fullsized":"2024-05-08T23:41:29.529076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5509","last_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_deep_scrub":"23'5509","last_deep_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.529050+0000","objects_scrubbed":0,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:10.031582+0000","scrub_duration":0.044696347999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6039","reported_seq":6055,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.042831+0000","last_change":"2024-05-08T23:32:10.351923+0000","last_active":"2024-05-08T23:41:12.042831+0000","last_peered":"2024-05-08T23:41:12.042831+0000","last_clean":"2024-05-08T23:41:12.042831+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:41:12.042831+0000","last_undegraded":"2024-05-08T23:41:12.042831+0000","last_fullsized":"2024-05-08T23:41:12.042831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6680","reported_seq":6696,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.060349+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:41:12.060349+0000","last_peered":"2024-05-08T23:41:12.060349+0000","last_clean":"2024-05-08T23:41:12.060349+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:41:12.060349+0000","last_undegraded":"2024-05-08T23:41:12.060349+0000","last_fullsized":"2024-05-08T23:41:12.060349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6680,"log_dups_size":0,"ondisk_log_size":6680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6521","reported_seq":6538,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:16.119070+0000","last_change":"2024-05-08T23:32:10.356351+0000","last_active":"2024-05-08T23:41:16.119070+0000","last_peered":"2024-05-08T23:41:16.119070+0000","last_clean":"2024-05-08T23:41:16.119070+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:41:16.119070+0000","last_undegraded":"2024-05-08T23:41:16.119070+0000","last_fullsized":"2024-05-08T23:41:16.119070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6521,"log_dups_size":0,"ondisk_log_size":6521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5781","reported_seq":5806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:33.687352+0000","last_change":"2024-05-08T23:41:33.687507+0000","last_active":"2024-05-08T23:41:33.687352+0000","last_peered":"2024-05-08T23:41:33.687352+0000","last_clean":"2024-05-08T23:41:33.687352+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:41:33.687352+0000","last_undegraded":"2024-05-08T23:41:33.687352+0000","last_fullsized":"2024-05-08T23:41:33.687352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5781","last_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_deep_scrub":"23'5781","last_deep_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_clean_scrub_stamp":"2024-05-08T23:41:33.687326+0000","objects_scrubbed":0,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:58.223805+0000","scrub_duration":0.028884346000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7981","reported_seq":8009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.740206+0000","last_change":"2024-05-08T23:41:28.740206+0000","last_active":"2024-05-08T23:41:28.740206+0000","last_peered":"2024-05-08T23:41:28.740206+0000","last_clean":"2024-05-08T23:41:28.740206+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:41:28.740206+0000","last_undegraded":"2024-05-08T23:41:28.740206+0000","last_fullsized":"2024-05-08T23:41:28.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7981","last_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_deep_scrub":"23'7981","last_deep_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.740034+0000","objects_scrubbed":0,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:13.685241+0000","scrub_duration":0.017426704000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5310","reported_seq":5338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.620114+0000","last_change":"2024-05-08T23:41:28.620267+0000","last_active":"2024-05-08T23:41:28.620114+0000","last_peered":"2024-05-08T23:41:28.620114+0000","last_clean":"2024-05-08T23:41:28.620114+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:41:28.620114+0000","last_undegraded":"2024-05-08T23:41:28.620114+0000","last_fullsized":"2024-05-08T23:41:28.620114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5310","last_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_deep_scrub":"23'5310","last_deep_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.620084+0000","objects_scrubbed":0,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:05.218724+0000","scrub_duration":0.024435924000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6330","reported_seq":6347,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.562842+0000","last_change":"2024-05-08T23:32:10.349600+0000","last_active":"2024-05-08T23:41:19.562842+0000","last_peered":"2024-05-08T23:41:19.562842+0000","last_clean":"2024-05-08T23:41:19.562842+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:41:19.562842+0000","last_undegraded":"2024-05-08T23:41:19.562842+0000","last_fullsized":"2024-05-08T23:41:19.562842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6330,"log_dups_size":0,"ondisk_log_size":6330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'386","reported_seq":415,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.402088+0000","last_change":"2024-05-08T23:41:22.402198+0000","last_active":"2024-05-08T23:41:22.402088+0000","last_peered":"2024-05-08T23:41:22.402088+0000","last_clean":"2024-05-08T23:41:22.402088+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:41:22.402088+0000","last_undegraded":"2024-05-08T23:41:22.402088+0000","last_fullsized":"2024-05-08T23:41:22.402088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'386","last_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_deep_scrub":"23'386","last_deep_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.402063+0000","objects_scrubbed":3,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:40.356038+0000","scrub_duration":0.050272126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6847","reported_seq":6873,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.635082+0000","last_change":"2024-05-08T23:41:30.635228+0000","last_active":"2024-05-08T23:41:30.635082+0000","last_peered":"2024-05-08T23:41:30.635082+0000","last_clean":"2024-05-08T23:41:30.635082+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:41:30.635082+0000","last_undegraded":"2024-05-08T23:41:30.635082+0000","last_fullsized":"2024-05-08T23:41:30.635082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6847","last_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_deep_scrub":"23'6847","last_deep_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.635055+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:39.371481+0000","scrub_duration":0.022460726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.929415+0000","last_change":"2024-05-08T23:41:18.929514+0000","last_active":"2024-05-08T23:41:18.929415+0000","last_peered":"2024-05-08T23:41:18.929415+0000","last_clean":"2024-05-08T23:41:18.929415+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:41:18.929415+0000","last_undegraded":"2024-05-08T23:41:18.929415+0000","last_fullsized":"2024-05-08T23:41:18.929415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.929388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.117796+0000","scrub_duration":0.0057181410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7917","reported_seq":7942,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.069649+0000","last_change":"2024-05-08T23:32:59.859469+0000","last_active":"2024-05-08T23:41:12.069649+0000","last_peered":"2024-05-08T23:41:12.069649+0000","last_clean":"2024-05-08T23:41:12.069649+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:41:12.069649+0000","last_undegraded":"2024-05-08T23:41:12.069649+0000","last_fullsized":"2024-05-08T23:41:12.069649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'10","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.842104+0000","last_change":"2024-05-08T23:32:09.889232+0000","last_active":"2024-05-08T23:40:50.842104+0000","last_peered":"2024-05-08T23:40:50.842104+0000","last_clean":"2024-05-08T23:40:50.842104+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:40:50.842104+0000","last_undegraded":"2024-05-08T23:40:50.842104+0000","last_fullsized":"2024-05-08T23:40:50.842104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4528","reported_seq":4544,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.137874+0000","last_change":"2024-05-08T23:32:10.357466+0000","last_active":"2024-05-08T23:41:12.137874+0000","last_peered":"2024-05-08T23:41:12.137874+0000","last_clean":"2024-05-08T23:41:12.137874+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:41:12.137874+0000","last_undegraded":"2024-05-08T23:41:12.137874+0000","last_fullsized":"2024-05-08T23:41:12.137874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":4528,"log_dups_size":0,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'7","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.806630+0000","last_change":"2024-05-08T23:41:30.806782+0000","last_active":"2024-05-08T23:41:30.806630+0000","last_peered":"2024-05-08T23:41:30.806630+0000","last_clean":"2024-05-08T23:41:30.806630+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:41:30.806630+0000","last_undegraded":"2024-05-08T23:41:30.806630+0000","last_fullsized":"2024-05-08T23:41:30.806630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.806600+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:00.816663+0000","scrub_duration":0.0035991980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7912","reported_seq":7928,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.153925+0000","last_change":"2024-05-08T23:32:10.357419+0000","last_active":"2024-05-08T23:41:12.153925+0000","last_peered":"2024-05-08T23:41:12.153925+0000","last_clean":"2024-05-08T23:41:12.153925+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:41:12.153925+0000","last_undegraded":"2024-05-08T23:41:12.153925+0000","last_fullsized":"2024-05-08T23:41:12.153925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1097","reported_seq":1125,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.879281+0000","last_change":"2024-05-08T23:41:21.879390+0000","last_active":"2024-05-08T23:41:21.879281+0000","last_peered":"2024-05-08T23:41:21.879281+0000","last_clean":"2024-05-08T23:41:21.879281+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:41:21.879281+0000","last_undegraded":"2024-05-08T23:41:21.879281+0000","last_fullsized":"2024-05-08T23:41:21.879281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'1097","last_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_deep_scrub":"23'1097","last_deep_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.879252+0000","objects_scrubbed":1,"log_size":1097,"log_dups_size":0,"ondisk_log_size":1097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:08.291506+0000","scrub_duration":0.026393197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5451","reported_seq":5478,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.439335+0000","last_change":"2024-05-08T23:41:32.439484+0000","last_active":"2024-05-08T23:41:32.439335+0000","last_peered":"2024-05-08T23:41:32.439335+0000","last_clean":"2024-05-08T23:41:32.439335+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:41:32.439335+0000","last_undegraded":"2024-05-08T23:41:32.439335+0000","last_fullsized":"2024-05-08T23:41:32.439335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5451","last_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_deep_scrub":"23'5451","last_deep_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.439309+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:57.926373+0000","scrub_duration":0.032867014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'27","reported_seq":55,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.226600+0000","last_change":"2024-05-08T23:41:23.226727+0000","last_active":"2024-05-08T23:41:23.226600+0000","last_peered":"2024-05-08T23:41:23.226600+0000","last_clean":"2024-05-08T23:41:23.226600+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:41:23.226600+0000","last_undegraded":"2024-05-08T23:41:23.226600+0000","last_fullsized":"2024-05-08T23:41:23.226600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'27","last_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_deep_scrub":"23'27","last_deep_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.226573+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:14.740811+0000","scrub_duration":0.0030473599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5065","reported_seq":5092,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.218177+0000","last_change":"2024-05-08T23:41:30.218312+0000","last_active":"2024-05-08T23:41:30.218177+0000","last_peered":"2024-05-08T23:41:30.218177+0000","last_clean":"2024-05-08T23:41:30.218177+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:41:30.218177+0000","last_undegraded":"2024-05-08T23:41:30.218177+0000","last_fullsized":"2024-05-08T23:41:30.218177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5065","last_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_deep_scrub":"23'5065","last_deep_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.218148+0000","objects_scrubbed":0,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:10.065500+0000","scrub_duration":0.024008964000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.922945+0000","last_change":"2024-05-08T23:41:27.923055+0000","last_active":"2024-05-08T23:41:27.922945+0000","last_peered":"2024-05-08T23:41:27.922945+0000","last_clean":"2024-05-08T23:41:27.922945+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:41:27.922945+0000","last_undegraded":"2024-05-08T23:41:27.922945+0000","last_fullsized":"2024-05-08T23:41:27.922945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.922913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:51.738030+0000","scrub_duration":0.0063254560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6601","reported_seq":6626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.688541+0000","last_change":"2024-05-08T23:41:32.688688+0000","last_active":"2024-05-08T23:41:32.688541+0000","last_peered":"2024-05-08T23:41:32.688541+0000","last_clean":"2024-05-08T23:41:32.688541+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:41:32.688541+0000","last_undegraded":"2024-05-08T23:41:32.688541+0000","last_fullsized":"2024-05-08T23:41:32.688541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6601","last_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_deep_scrub":"23'6601","last_deep_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.688513+0000","objects_scrubbed":0,"log_size":6601,"log_dups_size":0,"ondisk_log_size":6601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:26.541886+0000","scrub_duration":0.031381991999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:26.794388+0000","last_change":"2024-05-08T23:41:26.794830+0000","last_active":"2024-05-08T23:41:26.794388+0000","last_peered":"2024-05-08T23:41:26.794388+0000","last_clean":"2024-05-08T23:41:26.794388+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:41:26.794388+0000","last_undegraded":"2024-05-08T23:41:26.794388+0000","last_fullsized":"2024-05-08T23:41:26.794388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_clean_scrub_stamp":"2024-05-08T23:41:26.794365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:55.135134+0000","scrub_duration":0.001702903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7591","reported_seq":7607,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.067402+0000","last_change":"2024-05-08T23:32:10.349876+0000","last_active":"2024-05-08T23:41:12.067402+0000","last_peered":"2024-05-08T23:41:12.067402+0000","last_clean":"2024-05-08T23:41:12.067402+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:41:12.067402+0000","last_undegraded":"2024-05-08T23:41:12.067402+0000","last_fullsized":"2024-05-08T23:41:12.067402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:40:50.858039+0000","last_change":"2024-05-08T23:32:09.886347+0000","last_active":"2024-05-08T23:40:50.858039+0000","last_peered":"2024-05-08T23:40:50.858039+0000","last_clean":"2024-05-08T23:40:50.858039+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:40:50.858039+0000","last_undegraded":"2024-05-08T23:40:50.858039+0000","last_fullsized":"2024-05-08T23:40:50.858039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:08.861650+0000","last_clean_scrub_stamp":"2024-05-08T23:32:08.861650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6503","reported_seq":6530,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.781761+0000","last_change":"2024-05-08T23:41:29.781900+0000","last_active":"2024-05-08T23:41:29.781761+0000","last_peered":"2024-05-08T23:41:29.781761+0000","last_clean":"2024-05-08T23:41:29.781761+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:41:29.781761+0000","last_undegraded":"2024-05-08T23:41:29.781761+0000","last_fullsized":"2024-05-08T23:41:29.781761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6503","last_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_deep_scrub":"23'6503","last_deep_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.781726+0000","objects_scrubbed":0,"log_size":6503,"log_dups_size":0,"ondisk_log_size":6503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:40:01.229286+0000","scrub_duration":0.043658807000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.653793+0000","last_change":"2024-05-08T23:41:21.653942+0000","last_active":"2024-05-08T23:41:21.653793+0000","last_peered":"2024-05-08T23:41:21.653793+0000","last_clean":"2024-05-08T23:41:21.653793+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:41:21.653793+0000","last_undegraded":"2024-05-08T23:41:21.653793+0000","last_fullsized":"2024-05-08T23:41:21.653793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.653751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:19.453466+0000","scrub_duration":0.0043470959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5821","reported_seq":5856,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:37.117388+0000","last_change":"2024-05-08T23:41:37.117531+0000","last_active":"2024-05-08T23:41:37.117388+0000","last_peered":"2024-05-08T23:41:37.117388+0000","last_clean":"2024-05-08T23:41:37.117388+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:41:37.117388+0000","last_undegraded":"2024-05-08T23:41:37.117388+0000","last_fullsized":"2024-05-08T23:41:37.117388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5821","last_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_deep_scrub":"23'5821","last_deep_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_clean_scrub_stamp":"2024-05-08T23:41:37.117353+0000","objects_scrubbed":0,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:31.821747+0000","scrub_duration":0.021732336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'19","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:34.220458+0000","last_change":"2024-05-08T23:41:34.220610+0000","last_active":"2024-05-08T23:41:34.220458+0000","last_peered":"2024-05-08T23:41:34.220458+0000","last_clean":"2024-05-08T23:41:34.220458+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:41:34.220458+0000","last_undegraded":"2024-05-08T23:41:34.220458+0000","last_fullsized":"2024-05-08T23:41:34.220458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_clean_scrub_stamp":"2024-05-08T23:41:34.220429+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:47.942494+0000","scrub_duration":0.017072019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8452","reported_seq":8468,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.158657+0000","last_change":"2024-05-08T23:32:10.350649+0000","last_active":"2024-05-08T23:41:12.158657+0000","last_peered":"2024-05-08T23:41:12.158657+0000","last_clean":"2024-05-08T23:41:12.158657+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:41:12.158657+0000","last_undegraded":"2024-05-08T23:41:12.158657+0000","last_fullsized":"2024-05-08T23:41:12.158657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8452,"log_dups_size":0,"ondisk_log_size":8452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'226","reported_seq":254,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.134147+0000","last_change":"2024-05-08T23:41:29.134261+0000","last_active":"2024-05-08T23:41:29.134147+0000","last_peered":"2024-05-08T23:41:29.134147+0000","last_clean":"2024-05-08T23:41:29.134147+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:41:29.134147+0000","last_undegraded":"2024-05-08T23:41:29.134147+0000","last_fullsized":"2024-05-08T23:41:29.134147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'226","last_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_deep_scrub":"23'226","last_deep_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.134118+0000","objects_scrubbed":4,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:53.545220+0000","scrub_duration":0.018859172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6156","reported_seq":6172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.140465+0000","last_change":"2024-05-08T23:32:10.360644+0000","last_active":"2024-05-08T23:41:12.140465+0000","last_peered":"2024-05-08T23:41:12.140465+0000","last_clean":"2024-05-08T23:41:12.140465+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:41:12.140465+0000","last_undegraded":"2024-05-08T23:41:12.140465+0000","last_fullsized":"2024-05-08T23:41:12.140465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:20.387410+0000","last_change":"2024-05-08T23:41:20.387410+0000","last_active":"2024-05-08T23:41:20.387410+0000","last_peered":"2024-05-08T23:41:20.387410+0000","last_clean":"2024-05-08T23:41:20.387410+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:41:20.387410+0000","last_undegraded":"2024-05-08T23:41:20.387410+0000","last_fullsized":"2024-05-08T23:41:20.387410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_clean_scrub_stamp":"2024-05-08T23:41:20.387268+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:26.731831+0000","scrub_duration":0.0096185279999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5984","reported_seq":6001,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:15.549155+0000","last_change":"2024-05-08T23:32:10.350809+0000","last_active":"2024-05-08T23:41:15.549155+0000","last_peered":"2024-05-08T23:41:15.549155+0000","last_clean":"2024-05-08T23:41:15.549155+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:41:15.549155+0000","last_undegraded":"2024-05-08T23:41:15.549155+0000","last_fullsized":"2024-05-08T23:41:15.549155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.770600+0000","last_change":"2024-05-08T23:41:27.770600+0000","last_active":"2024-05-08T23:41:27.770600+0000","last_peered":"2024-05-08T23:41:27.770600+0000","last_clean":"2024-05-08T23:41:27.770600+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:41:27.770600+0000","last_undegraded":"2024-05-08T23:41:27.770600+0000","last_fullsized":"2024-05-08T23:41:27.770600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.770437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:39.778646+0000","scrub_duration":0.0041469519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7325","reported_seq":7341,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.167891+0000","last_change":"2024-05-08T23:32:10.358904+0000","last_active":"2024-05-08T23:41:12.167891+0000","last_peered":"2024-05-08T23:41:12.167891+0000","last_clean":"2024-05-08T23:41:12.167891+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:41:12.167891+0000","last_undegraded":"2024-05-08T23:41:12.167891+0000","last_fullsized":"2024-05-08T23:41:12.167891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7325,"log_dups_size":0,"ondisk_log_size":7325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.183501+0000","last_change":"2024-05-08T23:41:25.183501+0000","last_active":"2024-05-08T23:41:25.183501+0000","last_peered":"2024-05-08T23:41:25.183501+0000","last_clean":"2024-05-08T23:41:25.183501+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:41:25.183501+0000","last_undegraded":"2024-05-08T23:41:25.183501+0000","last_fullsized":"2024-05-08T23:41:25.183501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.183315+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:43.807059+0000","scrub_duration":0.001637573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7017","reported_seq":7034,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.979142+0000","last_change":"2024-05-08T23:32:10.354076+0000","last_active":"2024-05-08T23:41:18.979142+0000","last_peered":"2024-05-08T23:41:18.979142+0000","last_clean":"2024-05-08T23:41:18.979142+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:41:18.979142+0000","last_undegraded":"2024-05-08T23:41:18.979142+0000","last_fullsized":"2024-05-08T23:41:18.979142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7017,"log_dups_size":0,"ondisk_log_size":7017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:17.552762+0000","last_change":"2024-05-08T23:41:17.552762+0000","last_active":"2024-05-08T23:41:17.552762+0000","last_peered":"2024-05-08T23:41:17.552762+0000","last_clean":"2024-05-08T23:41:17.552762+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:41:17.552762+0000","last_undegraded":"2024-05-08T23:41:17.552762+0000","last_fullsized":"2024-05-08T23:41:17.552762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_clean_scrub_stamp":"2024-05-08T23:41:17.552644+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:34.431421+0000","scrub_duration":0.0077641810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6611","reported_seq":6628,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.562771+0000","last_change":"2024-05-08T23:32:10.358936+0000","last_active":"2024-05-08T23:41:19.562771+0000","last_peered":"2024-05-08T23:41:19.562771+0000","last_clean":"2024-05-08T23:41:19.562771+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:41:19.562771+0000","last_undegraded":"2024-05-08T23:41:19.562771+0000","last_fullsized":"2024-05-08T23:41:19.562771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.375277+0000","last_change":"2024-05-08T23:41:18.375277+0000","last_active":"2024-05-08T23:41:18.375277+0000","last_peered":"2024-05-08T23:41:18.375277+0000","last_clean":"2024-05-08T23:41:18.375277+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:41:18.375277+0000","last_undegraded":"2024-05-08T23:41:18.375277+0000","last_fullsized":"2024-05-08T23:41:18.375277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.375124+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:49.600961+0000","scrub_duration":0.017542283999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6076","reported_seq":6101,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.821861+0000","last_change":"2024-05-08T23:33:00.026523+0000","last_active":"2024-05-08T23:41:18.821861+0000","last_peered":"2024-05-08T23:41:18.821861+0000","last_clean":"2024-05-08T23:41:18.821861+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:41:18.821861+0000","last_undegraded":"2024-05-08T23:41:18.821861+0000","last_fullsized":"2024-05-08T23:41:18.821861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.208505+0000","last_change":"2024-05-08T23:41:22.208635+0000","last_active":"2024-05-08T23:41:22.208505+0000","last_peered":"2024-05-08T23:41:22.208505+0000","last_clean":"2024-05-08T23:41:22.208505+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:41:22.208505+0000","last_undegraded":"2024-05-08T23:41:22.208505+0000","last_fullsized":"2024-05-08T23:41:22.208505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.208478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:19.780610+0000","scrub_duration":0.0086337930000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5813","reported_seq":5830,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.110338+0000","last_change":"2024-05-08T23:32:10.351053+0000","last_active":"2024-05-08T23:41:12.110338+0000","last_peered":"2024-05-08T23:41:12.110338+0000","last_clean":"2024-05-08T23:41:12.110338+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:41:12.110338+0000","last_undegraded":"2024-05-08T23:41:12.110338+0000","last_fullsized":"2024-05-08T23:41:12.110338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.793158+0000","last_change":"2024-05-08T23:41:25.793278+0000","last_active":"2024-05-08T23:41:25.793158+0000","last_peered":"2024-05-08T23:41:25.793158+0000","last_clean":"2024-05-08T23:41:25.793158+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:41:25.793158+0000","last_undegraded":"2024-05-08T23:41:25.793158+0000","last_fullsized":"2024-05-08T23:41:25.793158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.793133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:17.035745+0000","scrub_duration":0.006691158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6336","reported_seq":6353,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.055620+0000","last_change":"2024-05-08T23:32:10.350539+0000","last_active":"2024-05-08T23:41:12.055620+0000","last_peered":"2024-05-08T23:41:12.055620+0000","last_clean":"2024-05-08T23:41:12.055620+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:41:12.055620+0000","last_undegraded":"2024-05-08T23:41:12.055620+0000","last_fullsized":"2024-05-08T23:41:12.055620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6281","reported_seq":6298,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.063409+0000","last_change":"2024-05-08T23:32:10.344834+0000","last_active":"2024-05-08T23:41:12.063409+0000","last_peered":"2024-05-08T23:41:12.063409+0000","last_clean":"2024-05-08T23:41:12.063409+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:41:12.063409+0000","last_undegraded":"2024-05-08T23:41:12.063409+0000","last_fullsized":"2024-05-08T23:41:12.063409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'28","reported_seq":81,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.579643+0000","last_change":"2024-05-08T23:41:25.579753+0000","last_active":"2024-05-08T23:41:25.579643+0000","last_peered":"2024-05-08T23:41:25.579643+0000","last_clean":"2024-05-08T23:41:25.579643+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:41:25.579643+0000","last_undegraded":"2024-05-08T23:41:25.579643+0000","last_fullsized":"2024-05-08T23:41:25.579643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'28","last_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_deep_scrub":"23'28","last_deep_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.579617+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:59.604194+0000","scrub_duration":0.010805047999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6855","reported_seq":6883,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.995720+0000","last_change":"2024-05-08T23:41:28.995863+0000","last_active":"2024-05-08T23:41:28.995720+0000","last_peered":"2024-05-08T23:41:28.995720+0000","last_clean":"2024-05-08T23:41:28.995720+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:41:28.995720+0000","last_undegraded":"2024-05-08T23:41:28.995720+0000","last_fullsized":"2024-05-08T23:41:28.995720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6855","last_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_deep_scrub":"23'6855","last_deep_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.995692+0000","objects_scrubbed":0,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:54.884404+0000","scrub_duration":0.036366062999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.587984+0000","last_change":"2024-05-08T23:41:19.588138+0000","last_active":"2024-05-08T23:41:19.587984+0000","last_peered":"2024-05-08T23:41:19.587984+0000","last_clean":"2024-05-08T23:41:19.587984+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:41:19.587984+0000","last_undegraded":"2024-05-08T23:41:19.587984+0000","last_fullsized":"2024-05-08T23:41:19.587984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.587945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:52.664086+0000","scrub_duration":0.012341421999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.816792+0000","last_change":"2024-05-08T23:41:23.816911+0000","last_active":"2024-05-08T23:41:23.816792+0000","last_peered":"2024-05-08T23:41:23.816792+0000","last_clean":"2024-05-08T23:41:23.816792+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:41:23.816792+0000","last_undegraded":"2024-05-08T23:41:23.816792+0000","last_fullsized":"2024-05-08T23:41:23.816792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.816760+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:47.733691+0000","scrub_duration":0.0062194889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6396","reported_seq":6424,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.256539+0000","last_change":"2024-05-08T23:41:27.256691+0000","last_active":"2024-05-08T23:41:27.256539+0000","last_peered":"2024-05-08T23:41:27.256539+0000","last_clean":"2024-05-08T23:41:27.256539+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:41:27.256539+0000","last_undegraded":"2024-05-08T23:41:27.256539+0000","last_fullsized":"2024-05-08T23:41:27.256539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6396","last_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_deep_scrub":"23'6396","last_deep_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.256513+0000","objects_scrubbed":0,"log_size":6396,"log_dups_size":0,"ondisk_log_size":6396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:10.352345+0000","scrub_duration":0.028481303999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.854006+0000","last_change":"2024-05-08T23:41:24.854120+0000","last_active":"2024-05-08T23:41:24.854006+0000","last_peered":"2024-05-08T23:41:24.854006+0000","last_clean":"2024-05-08T23:41:24.854006+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:41:24.854006+0000","last_undegraded":"2024-05-08T23:41:24.854006+0000","last_fullsized":"2024-05-08T23:41:24.854006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.853976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:46.648846+0000","scrub_duration":0.0020858780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.762775+0000","last_change":"2024-05-08T23:41:28.762886+0000","last_active":"2024-05-08T23:41:28.762775+0000","last_peered":"2024-05-08T23:41:28.762775+0000","last_clean":"2024-05-08T23:41:28.762775+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:41:28.762775+0000","last_undegraded":"2024-05-08T23:41:28.762775+0000","last_fullsized":"2024-05-08T23:41:28.762775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.762740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:30.867016+0000","scrub_duration":0.0025176629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7015","reported_seq":7043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:35.290975+0000","last_change":"2024-05-08T23:41:35.291126+0000","last_active":"2024-05-08T23:41:35.290975+0000","last_peered":"2024-05-08T23:41:35.290975+0000","last_clean":"2024-05-08T23:41:35.290975+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:41:35.290975+0000","last_undegraded":"2024-05-08T23:41:35.290975+0000","last_fullsized":"2024-05-08T23:41:35.290975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7015","last_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_deep_scrub":"23'7015","last_deep_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_clean_scrub_stamp":"2024-05-08T23:41:35.290948+0000","objects_scrubbed":0,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:19.726891+0000","scrub_duration":0.038503554000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'87","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.199741+0000","last_change":"2024-05-08T23:41:18.199836+0000","last_active":"2024-05-08T23:41:18.199741+0000","last_peered":"2024-05-08T23:41:18.199741+0000","last_clean":"2024-05-08T23:41:18.199741+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:41:18.199741+0000","last_undegraded":"2024-05-08T23:41:18.199741+0000","last_fullsized":"2024-05-08T23:41:18.199741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'87","last_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_deep_scrub":"23'87","last_deep_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.199711+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:53.571638+0000","scrub_duration":0.0083734289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6431","reported_seq":6448,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051664+0000","last_change":"2024-05-08T23:32:10.362244+0000","last_active":"2024-05-08T23:41:12.051664+0000","last_peered":"2024-05-08T23:41:12.051664+0000","last_clean":"2024-05-08T23:41:12.051664+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:41:12.051664+0000","last_undegraded":"2024-05-08T23:41:12.051664+0000","last_fullsized":"2024-05-08T23:41:12.051664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'9","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.238040+0000","last_change":"2024-05-08T23:41:21.238150+0000","last_active":"2024-05-08T23:41:21.238040+0000","last_peered":"2024-05-08T23:41:21.238040+0000","last_clean":"2024-05-08T23:41:21.238040+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:41:21.238040+0000","last_undegraded":"2024-05-08T23:41:21.238040+0000","last_fullsized":"2024-05-08T23:41:21.238040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9","last_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_deep_scrub":"23'9","last_deep_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.238014+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:43.382177+0000","scrub_duration":0.013372182999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5458","reported_seq":5475,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.040763+0000","last_change":"2024-05-08T23:32:10.356025+0000","last_active":"2024-05-08T23:41:12.040763+0000","last_peered":"2024-05-08T23:41:12.040763+0000","last_clean":"2024-05-08T23:41:12.040763+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:41:12.040763+0000","last_undegraded":"2024-05-08T23:41:12.040763+0000","last_fullsized":"2024-05-08T23:41:12.040763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.864949+0000","last_change":"2024-05-08T23:41:36.865073+0000","last_active":"2024-05-08T23:41:36.864949+0000","last_peered":"2024-05-08T23:41:36.864949+0000","last_clean":"2024-05-08T23:41:36.864949+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:41:36.864949+0000","last_undegraded":"2024-05-08T23:41:36.864949+0000","last_fullsized":"2024-05-08T23:41:36.864949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.864920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:52.653980+0000","scrub_duration":0.0019745399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5844","reported_seq":5872,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.531828+0000","last_change":"2024-05-08T23:41:27.531961+0000","last_active":"2024-05-08T23:41:27.531828+0000","last_peered":"2024-05-08T23:41:27.531828+0000","last_clean":"2024-05-08T23:41:27.531828+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:41:27.531828+0000","last_undegraded":"2024-05-08T23:41:27.531828+0000","last_fullsized":"2024-05-08T23:41:27.531828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5844","last_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_deep_scrub":"23'5844","last_deep_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.531800+0000","objects_scrubbed":0,"log_size":5844,"log_dups_size":0,"ondisk_log_size":5844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:04.924146+0000","scrub_duration":0.023550181999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.185501+0000","last_change":"2024-05-08T23:41:25.185636+0000","last_active":"2024-05-08T23:41:25.185501+0000","last_peered":"2024-05-08T23:41:25.185501+0000","last_clean":"2024-05-08T23:41:25.185501+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:41:25.185501+0000","last_undegraded":"2024-05-08T23:41:25.185501+0000","last_fullsized":"2024-05-08T23:41:25.185501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.185475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:27.661632+0000","scrub_duration":0.009407884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6488","reported_seq":6505,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.041984+0000","last_change":"2024-05-08T23:32:10.347741+0000","last_active":"2024-05-08T23:41:12.041984+0000","last_peered":"2024-05-08T23:41:12.041984+0000","last_clean":"2024-05-08T23:41:12.041984+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:41:12.041984+0000","last_undegraded":"2024-05-08T23:41:12.041984+0000","last_fullsized":"2024-05-08T23:41:12.041984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:31.230555+0000","last_change":"2024-05-08T23:41:31.230705+0000","last_active":"2024-05-08T23:41:31.230555+0000","last_peered":"2024-05-08T23:41:31.230555+0000","last_clean":"2024-05-08T23:41:31.230555+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:41:31.230555+0000","last_undegraded":"2024-05-08T23:41:31.230555+0000","last_fullsized":"2024-05-08T23:41:31.230555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_clean_scrub_stamp":"2024-05-08T23:41:31.230527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:50.791503+0000","scrub_duration":0.0091231479999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5341","reported_seq":5369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.944446+0000","last_change":"2024-05-08T23:41:30.944603+0000","last_active":"2024-05-08T23:41:30.944446+0000","last_peered":"2024-05-08T23:41:30.944446+0000","last_clean":"2024-05-08T23:41:30.944446+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:41:30.944446+0000","last_undegraded":"2024-05-08T23:41:30.944446+0000","last_fullsized":"2024-05-08T23:41:30.944446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5341","last_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_deep_scrub":"23'5341","last_deep_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.944419+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:37.870106+0000","scrub_duration":0.025456260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'687","reported_seq":716,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.835319+0000","last_change":"2024-05-08T23:41:25.835429+0000","last_active":"2024-05-08T23:41:25.835319+0000","last_peered":"2024-05-08T23:41:25.835319+0000","last_clean":"2024-05-08T23:41:25.835319+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:41:25.835319+0000","last_undegraded":"2024-05-08T23:41:25.835319+0000","last_fullsized":"2024-05-08T23:41:25.835319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'687","last_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_deep_scrub":"23'687","last_deep_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.835291+0000","objects_scrubbed":2,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:44.548476+0000","scrub_duration":0.021024537999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5938","reported_seq":5955,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.051200+0000","last_change":"2024-05-08T23:32:10.358135+0000","last_active":"2024-05-08T23:41:12.051200+0000","last_peered":"2024-05-08T23:41:12.051200+0000","last_clean":"2024-05-08T23:41:12.051200+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:41:12.051200+0000","last_undegraded":"2024-05-08T23:41:12.051200+0000","last_fullsized":"2024-05-08T23:41:12.051200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.520446+0000","last_change":"2024-05-08T23:41:24.520818+0000","last_active":"2024-05-08T23:41:24.520446+0000","last_peered":"2024-05-08T23:41:24.520446+0000","last_clean":"2024-05-08T23:41:24.520446+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:41:24.520446+0000","last_undegraded":"2024-05-08T23:41:24.520446+0000","last_fullsized":"2024-05-08T23:41:24.520446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.520417+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:30.649499+0000","scrub_duration":0.0021012050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6544","reported_seq":6572,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.866623+0000","last_change":"2024-05-08T23:41:19.866792+0000","last_active":"2024-05-08T23:41:19.866623+0000","last_peered":"2024-05-08T23:41:19.866623+0000","last_clean":"2024-05-08T23:41:19.866623+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:41:19.866623+0000","last_undegraded":"2024-05-08T23:41:19.866623+0000","last_fullsized":"2024-05-08T23:41:19.866623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6544","last_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_deep_scrub":"23'6544","last_deep_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.866594+0000","objects_scrubbed":0,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:33.883813+0000","scrub_duration":0.03064476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.262007+0000","last_change":"2024-05-08T23:41:28.262123+0000","last_active":"2024-05-08T23:41:28.262007+0000","last_peered":"2024-05-08T23:41:28.262007+0000","last_clean":"2024-05-08T23:41:28.262007+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:41:28.262007+0000","last_undegraded":"2024-05-08T23:41:28.262007+0000","last_fullsized":"2024-05-08T23:41:28.262007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.261980+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:56.437680+0000","scrub_duration":0.018641068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6824","reported_seq":6852,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.871008+0000","last_change":"2024-05-08T23:41:36.871129+0000","last_active":"2024-05-08T23:41:36.871008+0000","last_peered":"2024-05-08T23:41:36.871008+0000","last_clean":"2024-05-08T23:41:36.871008+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:41:36.871008+0000","last_undegraded":"2024-05-08T23:41:36.871008+0000","last_fullsized":"2024-05-08T23:41:36.871008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6824","last_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_deep_scrub":"23'6824","last_deep_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.870982+0000","objects_scrubbed":0,"log_size":6824,"log_dups_size":0,"ondisk_log_size":6824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:55.226753+0000","scrub_duration":0.026360792000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.492931+0000","last_change":"2024-05-08T23:41:19.492931+0000","last_active":"2024-05-08T23:41:19.492931+0000","last_peered":"2024-05-08T23:41:19.492931+0000","last_clean":"2024-05-08T23:41:19.492931+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:41:19.492931+0000","last_undegraded":"2024-05-08T23:41:19.492931+0000","last_fullsized":"2024-05-08T23:41:19.492931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.492750+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:25.643838+0000","scrub_duration":0.0040015720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7700","reported_seq":7717,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.562825+0000","last_change":"2024-05-08T23:32:10.356529+0000","last_active":"2024-05-08T23:41:19.562825+0000","last_peered":"2024-05-08T23:41:19.562825+0000","last_clean":"2024-05-08T23:41:19.562825+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:41:19.562825+0000","last_undegraded":"2024-05-08T23:41:19.562825+0000","last_fullsized":"2024-05-08T23:41:19.562825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7806","reported_seq":7823,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.081482+0000","last_change":"2024-05-08T23:32:10.354095+0000","last_active":"2024-05-08T23:41:12.081482+0000","last_peered":"2024-05-08T23:41:12.081482+0000","last_clean":"2024-05-08T23:41:12.081482+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:41:12.081482+0000","last_undegraded":"2024-05-08T23:41:12.081482+0000","last_fullsized":"2024-05-08T23:41:12.081482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7806,"log_dups_size":0,"ondisk_log_size":7806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6574","reported_seq":6602,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.586969+0000","last_change":"2024-05-08T23:41:27.587096+0000","last_active":"2024-05-08T23:41:27.586969+0000","last_peered":"2024-05-08T23:41:27.586969+0000","last_clean":"2024-05-08T23:41:27.586969+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:41:27.586969+0000","last_undegraded":"2024-05-08T23:41:27.586969+0000","last_fullsized":"2024-05-08T23:41:27.586969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6574","last_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_deep_scrub":"23'6574","last_deep_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.586941+0000","objects_scrubbed":0,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:35.348883+0000","scrub_duration":0.027947453000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8085","reported_seq":8102,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.170588+0000","last_change":"2024-05-08T23:32:10.352808+0000","last_active":"2024-05-08T23:41:12.170588+0000","last_peered":"2024-05-08T23:41:12.170588+0000","last_clean":"2024-05-08T23:41:12.170588+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:41:12.170588+0000","last_undegraded":"2024-05-08T23:41:12.170588+0000","last_fullsized":"2024-05-08T23:41:12.170588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5657","reported_seq":5674,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.097092+0000","last_change":"2024-05-08T23:32:10.350374+0000","last_active":"2024-05-08T23:41:12.097092+0000","last_peered":"2024-05-08T23:41:12.097092+0000","last_clean":"2024-05-08T23:41:12.097092+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:41:12.097092+0000","last_undegraded":"2024-05-08T23:41:12.097092+0000","last_fullsized":"2024-05-08T23:41:12.097092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6017","reported_seq":6034,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.073073+0000","last_change":"2024-05-08T23:32:10.354820+0000","last_active":"2024-05-08T23:41:12.073073+0000","last_peered":"2024-05-08T23:41:12.073073+0000","last_clean":"2024-05-08T23:41:12.073073+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:41:12.073073+0000","last_undegraded":"2024-05-08T23:41:12.073073+0000","last_fullsized":"2024-05-08T23:41:12.073073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7053","reported_seq":7070,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.066767+0000","last_change":"2024-05-08T23:32:10.348922+0000","last_active":"2024-05-08T23:41:12.066767+0000","last_peered":"2024-05-08T23:41:12.066767+0000","last_clean":"2024-05-08T23:41:12.066767+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:41:12.066767+0000","last_undegraded":"2024-05-08T23:41:12.066767+0000","last_fullsized":"2024-05-08T23:41:12.066767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5666","reported_seq":5683,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.152965+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:41:12.152965+0000","last_peered":"2024-05-08T23:41:12.152965+0000","last_clean":"2024-05-08T23:41:12.152965+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:41:12.152965+0000","last_undegraded":"2024-05-08T23:41:12.152965+0000","last_fullsized":"2024-05-08T23:41:12.152965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7698","reported_seq":7715,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:11.975204+0000","last_change":"2024-05-08T23:32:10.354163+0000","last_active":"2024-05-08T23:41:11.975204+0000","last_peered":"2024-05-08T23:41:11.975204+0000","last_clean":"2024-05-08T23:41:11.975204+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:41:11.975204+0000","last_undegraded":"2024-05-08T23:41:11.975204+0000","last_fullsized":"2024-05-08T23:41:11.975204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5557","reported_seq":5574,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.100041+0000","last_change":"2024-05-08T23:32:10.357074+0000","last_active":"2024-05-08T23:41:12.100041+0000","last_peered":"2024-05-08T23:41:12.100041+0000","last_clean":"2024-05-08T23:41:12.100041+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:41:12.100041+0000","last_undegraded":"2024-05-08T23:41:12.100041+0000","last_fullsized":"2024-05-08T23:41:12.100041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5557,"log_dups_size":0,"ondisk_log_size":5557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6393","reported_seq":6421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.942026+0000","last_change":"2024-05-08T23:41:32.942160+0000","last_active":"2024-05-08T23:41:32.942026+0000","last_peered":"2024-05-08T23:41:32.942026+0000","last_clean":"2024-05-08T23:41:32.942026+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:41:32.942026+0000","last_undegraded":"2024-05-08T23:41:32.942026+0000","last_fullsized":"2024-05-08T23:41:32.942026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6393","last_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_deep_scrub":"23'6393","last_deep_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.942001+0000","objects_scrubbed":0,"log_size":6393,"log_dups_size":0,"ondisk_log_size":6393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:28.602761+0000","scrub_duration":0.037578449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7241","reported_seq":7258,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:13.778962+0000","last_change":"2024-05-08T23:32:10.348032+0000","last_active":"2024-05-08T23:41:13.778962+0000","last_peered":"2024-05-08T23:41:13.778962+0000","last_clean":"2024-05-08T23:41:13.778962+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:41:13.778962+0000","last_undegraded":"2024-05-08T23:41:13.778962+0000","last_fullsized":"2024-05-08T23:41:13.778962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6810","reported_seq":6827,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:15.549263+0000","last_change":"2024-05-08T23:32:10.350519+0000","last_active":"2024-05-08T23:41:15.549263+0000","last_peered":"2024-05-08T23:41:15.549263+0000","last_clean":"2024-05-08T23:41:15.549263+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:41:15.549263+0000","last_undegraded":"2024-05-08T23:41:15.549263+0000","last_fullsized":"2024-05-08T23:41:15.549263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6394","reported_seq":6411,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:17.649441+0000","last_change":"2024-05-08T23:32:10.357823+0000","last_active":"2024-05-08T23:41:17.649441+0000","last_peered":"2024-05-08T23:41:17.649441+0000","last_clean":"2024-05-08T23:41:17.649441+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:41:17.649441+0000","last_undegraded":"2024-05-08T23:41:17.649441+0000","last_fullsized":"2024-05-08T23:41:17.649441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419416,"ondisk_log_size":419416,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293737,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2683,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34587474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2677,"ondisk_log_size":2677,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617404,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":604286,"kb_avail":93754436,"statfs":{"total":96636764160,"available":96004542464,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":618789323},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":8.0020000000000007,"15min":3.2799999999999998},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.9000000000000004,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.536,"5min":7.2050000000000001,"15min":3.1040000000000001},"min":{"1min":0.34699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":15.487,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.44600000000000001}]},{"osd":1,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":4.0140000000000002,"15min":1.996},"min":{"1min":0.40000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5350000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":1.2969999999999999,"5min":7.3179999999999996,"15min":3.161},"min":{"1min":0.36799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":7.9279999999999999,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.38100000000000001}]},{"osd":2,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":3.827,"15min":1.944},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.9589999999999996,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.609,"5min":4.7839999999999998,"15min":2.1850000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.375,"5min":26.084,"15min":26.084},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":6.6429999999999998,"15min":2.9350000000000001},"min":{"1min":0.45300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.2999999999999998,"5min":52.869,"15min":52.869},"last":0.498},{"interface":"front","average":{"1min":1.478,"5min":7.1310000000000002,"15min":2.911},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.35499999999999998},"max":{"1min":5.9299999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":4.4610000000000003,"15min":2.089},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.8109999999999999,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":1.129,"5min":3.5609999999999999,"15min":1.7310000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":7.8550000000000004,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":4.2480000000000002,"15min":2.1600000000000001},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.33100000000000002},"max":{"1min":10.327,"5min":25.116,"15min":63.732999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":1.2649999999999999,"5min":6.2969999999999997,"15min":3.028},"min":{"1min":0.46100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.5020000000000007,"5min":37.552999999999997,"15min":62.558},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":2.0070000000000001,"5min":6.8129999999999997,"15min":3.0409999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":13.154999999999999,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.7629999999999999,"5min":4.4379999999999997,"15min":2.056},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.143000000000001,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.52700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4710076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":6.3659999999999997,"15min":2.8140000000000001},"min":{"1min":0.36199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":8.0109999999999992,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":6.9820000000000002,"15min":3.1440000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.25800000000000001},"max":{"1min":3.9660000000000002,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.78800000000000003}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":5.9649999999999999,"15min":2.8879999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4629999999999992,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":5.2009999999999996,"15min":2.6269999999999998},"min":{"1min":0.43099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":4.0460000000000003,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":4.0869999999999997,"15min":2.1139999999999999},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.28599999999999998},"max":{"1min":4.0099999999999998,"5min":41.445,"15min":41.445},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":4.1420000000000003,"15min":2.085},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.0880000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.76400000000000001}]},{"osd":3,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":4.3869999999999996,"15min":2.3119999999999998},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.9060000000000006,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":6.5350000000000001,"15min":3.2010000000000001},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":8.8650000000000002,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":2.9929999999999999,"15min":1.466},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.069,"5min":20.913,"15min":20.913},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":3.2480000000000002,"15min":1.831},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.32600000000000001},"max":{"1min":4.1230000000000002,"5min":22.361000000000001,"15min":35.975999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":5.9729999999999999,"15min":2.7229999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.4199999999999999,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.4850000000000001,"5min":4.3899999999999997,"15min":2.081},"min":{"1min":0.373,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.3130000000000006,"5min":23.105,"15min":23.105},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":3.5179999999999998,"15min":2.1019999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.32300000000000001},"max":{"1min":16.905999999999999,"5min":24.16,"15min":33.463000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.74,"5min":5.5640000000000001,"15min":2.4199999999999999},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.433,"5min":45.805,"15min":45.805},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":5.8040000000000003,"15min":2.5310000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":3.6070000000000002,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":3.8109999999999999,"15min":2.1469999999999998},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":0.98799999999999999,"5min":39.283999999999999,"15min":48.844999999999999},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":3.7309999999999999,"15min":1.891},"min":{"1min":0.34200000000000003,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":1.2230000000000001,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":5.2709999999999999,"15min":2.3410000000000002},"min":{"1min":0.44,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":3.9390000000000001,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.40899999999999997}]},{"osd":3,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":6.6470000000000002,"15min":2.972},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.27700000000000002},"max":{"1min":1.1379999999999999,"5min":46.390999999999998,"15min":52.712000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":7.0099999999999998,"15min":3.4780000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32100000000000001},"max":{"1min":1.2929999999999999,"5min":53.265999999999998,"15min":61.832000000000001},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":5.3079999999999998,"15min":2.7050000000000001},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":6.8620000000000001,"5min":34.500999999999998,"15min":45.267000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":5.4020000000000001,"15min":2.6099999999999999},"min":{"1min":0.52200000000000002,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":3.7810000000000001,"5min":34.347999999999999,"15min":43.353999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":5.0700000000000003,"15min":2.6989999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.8200000000000003,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":6.5780000000000003,"15min":3.2839999999999998},"min":{"1min":0.60399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":5.8310000000000004,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.47099999999999997}]},{"osd":6,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":5.6740000000000004,"15min":3.0920000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.9780000000000002,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.97299999999999998,"5min":5.5620000000000003,"15min":2.9009999999999998},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":6.3860000000000001,"5min":37.802,"15min":42.603999999999999},"last":0.504}]},{"osd":7,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":7.7400000000000002,"15min":3.4689999999999999},"min":{"1min":0.504,"5min":0.42699999999999999,"15min":0.38100000000000001},"max":{"1min":4.6260000000000003,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":7.6180000000000003,"15min":3.7349999999999999},"min":{"1min":0.52800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.6630000000000003,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":5.5460000000000003,"15min":2.899},"min":{"1min":0.216,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.5140000000000002,"5min":47.241999999999997,"15min":101.899},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":4.5510000000000002,"15min":2.5579999999999998},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.2690000000000001,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.40899999999999997}]},{"osd":2,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":7.4269999999999996,"15min":3.5910000000000002},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.8660000000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":6.5880000000000001,"15min":3.3519999999999999},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":2.3519999999999999,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.501}]},{"osd":3,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.423,"5min":6.2320000000000002,"15min":3.383},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":24.239000000000001,"5min":59.796999999999997,"15min":101.929},"last":0.47199999999999998},{"interface":"front","average":{"1min":2.399,"5min":3.3879999999999999,"15min":2.3410000000000002},"min":{"1min":0.32300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":23.664999999999999,"5min":23.664999999999999,"15min":95.200999999999993},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":6.0060000000000002,"15min":3.2549999999999999},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.35399999999999998},"max":{"1min":6.9850000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.228,"5min":3.7610000000000001,"15min":2.3889999999999998},"min":{"1min":0.46600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.5709999999999997,"5min":19.163,"15min":44.545999999999999},"last":0.73299999999999998}]},{"osd":5,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.3410000000000002,"5min":6.1820000000000004,"15min":3.0310000000000001},"min":{"1min":0.44700000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":43.124000000000002,"5min":43.124000000000002,"15min":78.780000000000001},"last":0.626},{"interface":"front","average":{"1min":4.2110000000000003,"5min":5.843,"15min":3.0070000000000001},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":43.109000000000002,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.68899999999999995}]},{"osd":6,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":5.8769999999999998,"15min":3.351},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":11.282999999999999,"5min":45.744999999999997,"15min":101.869},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.284,"5min":5.0599999999999996,"15min":2.944},"min":{"1min":0.48199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":6.8890000000000002,"5min":38.545000000000002,"15min":78.647999999999996},"last":0.53500000000000003}]},{"osd":7,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.4509999999999996,"15min":2.9009999999999998},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":6.9580000000000002,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.482,"5min":3.7200000000000002,"15min":2.3279999999999998},"min":{"1min":0.48999999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.532,"5min":19.065000000000001,"15min":44.713999999999999},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":5.266,"15min":2.2570000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":9.6880000000000006,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.3129999999999999,"5min":4.6859999999999999,"15min":2.149},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.27700000000000002},"max":{"1min":9.7699999999999996,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.41199999999999998}]},{"osd":1,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":5.0739999999999998,"15min":2.5710000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":1.9159999999999999,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.435},{"interface":"front","average":{"1min":1.2330000000000001,"5min":5.4859999999999998,"15min":2.524},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.318},"max":{"1min":7.0469999999999997,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":7.5650000000000004,"15min":3.516},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27600000000000002},"max":{"1min":22.989999999999998,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.622,"5min":4.0359999999999996,"15min":2.0750000000000002},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":1.6659999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.1959999999999997,"15min":2.363},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":29.513999999999999,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.1339999999999999,"5min":3.8719999999999999,"15min":2.0150000000000001},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":7.0970000000000004,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":5.6399999999999997,"15min":2.8500000000000001},"min":{"1min":0.49199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":12.858000000000001,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.853,"5min":5.0960000000000001,"15min":2.371},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":12.723000000000001,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":4.8440000000000003,"15min":2.278},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":10.128,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.647,"5min":5.8680000000000003,"15min":2.605},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":9.3510000000000009,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":4.9199999999999999,"15min":2.3759999999999999},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.5679999999999996,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":2.2879999999999998,"5min":6.5650000000000004,"15min":2.96},"min":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.9250000000000007,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.58899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":5.181,"15min":2.605},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.22600000000000001},"max":{"1min":6.7359999999999998,"5min":53.103999999999999,"15min":67.305999999999997},"last":0.379},{"interface":"front","average":{"1min":1.5149999999999999,"5min":6.6479999999999997,"15min":3.464},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.592000000000001,"5min":39.784999999999997,"15min":106.934},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":6.1829999999999998,"15min":2.7589999999999999},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":14.182,"5min":76.628,"15min":76.628},"last":0.48199999999999998},{"interface":"front","average":{"1min":2.0030000000000001,"5min":5.4720000000000004,"15min":2.6099999999999999},"min":{"1min":0.39900000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":14.577,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.67900000000000005}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":7.0739999999999998,"15min":3.3050000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":65.519000000000005,"5min":65.519000000000005,"15min":65.519000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.375,"5min":4.75,"15min":2.4169999999999998},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.057,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":7.9180000000000001,"15min":3.5569999999999999},"min":{"1min":0.40799999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":65.575000000000003,"5min":65.575000000000003,"15min":65.575000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":3.726,"5min":7.548,"15min":3.137},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":31.77,"5min":68.238,"15min":68.238},"last":0.503}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.2990000000000004,"5min":6.6740000000000004,"15min":2.9620000000000002},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.38100000000000001},"max":{"1min":31.806000000000001,"5min":53.381,"15min":53.381},"last":0.49299999999999999},{"interface":"front","average":{"1min":3.157,"5min":6.7670000000000003,"15min":3.294},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":26.997,"5min":32.381,"15min":36.213999999999999},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.1429999999999998,"5min":8.0150000000000006,"15min":3.391},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":65.606999999999999,"5min":65.606999999999999,"15min":65.606999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.925,"5min":5.1440000000000001,"15min":2.464},"min":{"1min":0.57099999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":10.94,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9100000000000001,"5min":9.0250000000000004,"15min":3.8580000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":65.638999999999996,"5min":65.638999999999996,"15min":65.638999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.421,"5min":6.2400000000000002,"15min":2.9540000000000002},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.681,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":6.1509999999999998,"15min":2.8980000000000001},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.3620000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":2.9729999999999999,"5min":6.7389999999999999,"15min":3.109},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":10.945,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.82199999999999995}]},{"osd":1,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7050000000000001,"5min":5.7030000000000003,"15min":2.9729999999999999},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":24.324999999999999,"5min":35.807000000000002,"15min":40.901000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":5.6130000000000004,"5min":5.867,"15min":2.8839999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":47.481999999999999,"5min":47.481999999999999,"15min":47.481999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.9279999999999999,"5min":5.4900000000000002,"15min":2.871},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":13.356999999999999,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.80800000000000005},{"interface":"front","average":{"1min":2.7730000000000001,"5min":4.8339999999999996,"15min":2.5529999999999999},"min":{"1min":0.42199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":11.509,"5min":25.616,"15min":58.899000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.835,"15min":2.859},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.318},"max":{"1min":24.460999999999999,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.753},{"interface":"front","average":{"1min":4.3490000000000002,"5min":7.7370000000000001,"15min":3.4350000000000001},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":31.451000000000001,"5min":111.221,"15min":111.221},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":4.6660000000000004,"15min":2.266},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.7229999999999999,"5min":43.497999999999998,"15min":51.871000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.8460000000000001,"5min":4.4720000000000004,"15min":2.0409999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.7100000000000009,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2930000000000001,"5min":4.5890000000000004,"15min":2.0680000000000001},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.436999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":2.2069999999999999,"5min":4.548,"15min":2.3849999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":11.083,"5min":43.774999999999999,"15min":51.932000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":4.0289999999999999,"15min":1.9630000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":7.4790000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":2.6930000000000001,"5min":4.7949999999999999,"15min":2.4729999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":14.221,"5min":44.783000000000001,"15min":51.978000000000002},"last":0.628}]}]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":5.4900000000000002,"15min":2.7029999999999998},"min":{"1min":0.28599999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.4160000000000004,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.8300000000000001,"5min":5.2009999999999996,"15min":2.3119999999999998},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.30199999999999999},"max":{"1min":7.7690000000000001,"5min":44.439,"15min":44.439},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":4.9509999999999996,"15min":2.9039999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":7.2759999999999998,"5min":31.355,"15min":124.85599999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.347,"5min":4.8479999999999999,"15min":2.4580000000000002},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.317},"max":{"1min":7.327,"5min":31.385000000000002,"15min":41.060000000000002},"last":0.84199999999999997}]},{"osd":2,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":6.5110000000000001,"15min":2.9790000000000001},"min":{"1min":0.45600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.8650000000000002,"5min":44.247,"15min":44.247},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.7170000000000001,"5min":4.8680000000000003,"15min":2.3039999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":7.5350000000000001,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.79300000000000004}]},{"osd":3,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":5.0279999999999996,"5min":7.7690000000000001,"15min":3.1699999999999999},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":39.265000000000001,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.75700000000000001},{"interface":"front","average":{"1min":4.1520000000000001,"5min":7.9029999999999996,"15min":4.0640000000000001},"min":{"1min":0.46100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.193000000000001,"5min":43.343000000000004,"15min":49.991999999999997},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.1840000000000002,"15min":2.181},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":8.8119999999999994,"5min":43.305,"15min":43.305},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.4199999999999999,"5min":3.669,"15min":1.8049999999999999},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.3780000000000001,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.504}]},{"osd":6,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":5.4550000000000001,"15min":2.855},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":7.8280000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":2.0369999999999999,"5min":4.7329999999999997,"15min":2.306},"min":{"1min":0.45000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.2289999999999992,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":5.6580000000000004,"15min":2.532},"min":{"1min":0.44500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":15.589,"5min":43.262,"15min":43.262},"last":0.66100000000000003},{"interface":"front","average":{"1min":2.2690000000000001,"5min":6.3700000000000001,"15min":3.3159999999999998},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":10.456,"5min":44.018000000000001,"15min":80.302000000000007},"last":0.57599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4622048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:41:40.389 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.389 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.389 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.390 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.391 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-08T23:32:08.861650+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=8, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.392 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.393 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.394 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.395 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:41:40.396 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T23:41:51.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:41:51.721 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:00.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:42:00.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.544+0000 7f73be517700 1 Processor -- start 2024-05-08T23:42:00.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.544+0000 7f73be517700 1 -- start start 2024-05-08T23:42:00.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.544+0000 7f73be517700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73b80ce150 0x7f73b8153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:00.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73be517700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b815e6e0 con 0x7f73b80c9c30 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73be517700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80c9090 con 0x7f73b80cdd70 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73be517700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b80c3b60 con 0x7f73b80ce150 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b77fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73b80ce150 0x7f73b8153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73bcab4700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f73b80c9c30 0x7f73b8070110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:58568/0 (socket says 172.21.15.33:58568) 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73bcab4700 1 -- 172.21.15.33:0/52807289 learned_addr learned my addr 172.21.15.33:0/52807289 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b77fe700 1 -- 172.21.15.33:0/52807289 >> v1:172.21.15.42:6789/0 conn(0x7f73b80cdd70 legacy=0x7f73b80c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b77fe700 1 -- 172.21.15.33:0/52807289 >> v1:172.21.15.33:6789/0 conn(0x7f73b80c9c30 legacy=0x7f73b8070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b77fe700 1 -- 172.21.15.33:0/52807289 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b815db60 con 0x7f73b80ce150 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b77fe700 1 --2- 172.21.15.33:0/52807289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80ce150 0x7f73b8153fd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009870 tx=0x7f73ac009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1c5de1da413a92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b6ffd700 1 -- 172.21.15.33:0/52807289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011830 con 0x7f73b80ce150 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b6ffd700 1 -- 172.21.15.33:0/52807289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac011990 con 0x7f73b80ce150 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.545+0000 7f73b6ffd700 1 -- 172.21.15.33:0/52807289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011d20 con 0x7f73b80ce150 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 -- 172.21.15.33:0/52807289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80ce150 msgr2=0x7f73b8153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 --2- 172.21.15.33:0/52807289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80ce150 0x7f73b8153fd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009870 tx=0x7f73ac009b80 comp rx=0 tx=0).stop 2024-05-08T23:42:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 -- 172.21.15.33:0/52807289 shutdown_connections 2024-05-08T23:42:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 --2- 172.21.15.33:0/52807289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80ce150 0x7f73b8153fd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 -- 172.21.15.33:0/52807289 >> 172.21.15.33:0/52807289 conn(0x7f73b80666c0 msgr2=0x7f73b8066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 -- 172.21.15.33:0/52807289 shutdown_connections 2024-05-08T23:42:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.546+0000 7f73be517700 1 -- 172.21.15.33:0/52807289 wait complete. 2024-05-08T23:42:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.547+0000 7f73be517700 1 Processor -- start 2024-05-08T23:42:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 -- start start 2024-05-08T23:42:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b80cdd70 0x7f73b81efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b80ce150 0x7f73b806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b814b590 con 0x7f73b80c9c30 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b81e5ea0 con 0x7f73b80ce150 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73be517700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b81524b0 con 0x7f73b80cdd70 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.548+0000 7f73b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42208/0 (socket says 172.21.15.33:42208) 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 -- 172.21.15.33:0/1854321877 learned_addr learned my addr 172.21.15.33:0/1854321877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73bcab4700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b80ce150 0x7f73b806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b77fe700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b80cdd70 0x7f73b81efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 -- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b80cdd70 msgr2=0x7f73b81efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b80cdd70 0x7f73b81efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 -- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b80ce150 msgr2=0x7f73b806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b80ce150 0x7f73b806d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 -- 172.21.15.33:0/1854321877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0094a0 con 0x7f73b80c9c30 2024-05-08T23:42:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b7fff700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c510 tx=0x7f73a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80052d0 con 0x7f73b80c9c30 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8005430 con 0x7f73b80c9c30 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b81588c0 con 0x7f73b80c9c30 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a80038c0 con 0x7f73b80c9c30 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.549+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b81e6330 con 0x7f73b80c9c30 2024-05-08T23:42:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.550+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b80c40c0 con 0x7f73b80c9c30 2024-05-08T23:42:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.551+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f73a8003a20 con 0x7f73b80c9c30 2024-05-08T23:42:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.551+0000 7f73b4ff9700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 0x7f73a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.552+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f73a80889b0 con 0x7f73b80c9c30 2024-05-08T23:42:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.552+0000 7f73b77fe700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 0x7f73a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.552+0000 7f73b77fe700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 0x7f73a406d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0090b0 tx=0x7f73ac01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:00.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.556+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f73a8053990 con 0x7f73b80c9c30 2024-05-08T23:42:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.772+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73b815e400 con 0x7f73a406ac90 2024-05-08T23:42:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.781+0000 7f73b4ff9700 1 -- 172.21.15.33:0/1854321877 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292199 (secure 0 0 0) 0x7f73ac011c80 con 0x7f73a406ac90 2024-05-08T23:42:00.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:42:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 msgr2=0x7f73a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 0x7f73a406d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0090b0 tx=0x7f73ac01c040 comp rx=0 tx=0).stop 2024-05-08T23:42:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 msgr2=0x7f73b815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73a800c510 tx=0x7f73a800c8d0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 shutdown_connections 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f73a406ac90 0x7f73a406d150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73b80ce150 0x7f73b806d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73b80c9c30 0x7f73b815a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 --2- 172.21.15.33:0/1854321877 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f73b80cdd70 0x7f73b81efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 >> 172.21.15.33:0/1854321877 conn(0x7f73b80666c0 msgr2=0x7f73b80700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 shutdown_connections 2024-05-08T23:42:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:00.785+0000 7f73be517700 1 -- 172.21.15.33:0/1854321877 wait complete. 2024-05-08T23:42:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:42:00.796 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-08T23:41:59.507885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17884105,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320951,"num_write_kb":49019247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422213,"ondisk_log_size":422213,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4048792,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4011827,"kb_avail":750925928,"statfs":{"total":773094113280,"available":768948150272,"internally_reserved":0,"allocated":37773312,"data_stored":36472434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4108111416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003365"},"pg_stats":[{"pgid":"3.3f","version":"23'5702","reported_seq":5730,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:58.802399+0000","last_change":"2024-05-08T23:41:58.802542+0000","last_active":"2024-05-08T23:41:58.802399+0000","last_peered":"2024-05-08T23:41:58.802399+0000","last_clean":"2024-05-08T23:41:58.802399+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:41:58.802399+0000","last_undegraded":"2024-05-08T23:41:58.802399+0000","last_fullsized":"2024-05-08T23:41:58.802399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5702","last_scrub_stamp":"2024-05-08T23:41:58.802371+0000","last_deep_scrub":"23'5702","last_deep_scrub_stamp":"2024-05-08T23:41:58.802371+0000","last_clean_scrub_stamp":"2024-05-08T23:41:58.802371+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:45.817840+0000","scrub_duration":0.0094510960000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8392","reported_seq":8420,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.339055+0000","last_change":"2024-05-08T23:41:47.339196+0000","last_active":"2024-05-08T23:41:47.339055+0000","last_peered":"2024-05-08T23:41:47.339055+0000","last_clean":"2024-05-08T23:41:47.339055+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:41:47.339055+0000","last_undegraded":"2024-05-08T23:41:47.339055+0000","last_fullsized":"2024-05-08T23:41:47.339055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8392","last_scrub_stamp":"2024-05-08T23:41:47.339029+0000","last_deep_scrub":"23'8392","last_deep_scrub_stamp":"2024-05-08T23:41:47.339029+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.339029+0000","objects_scrubbed":0,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:35.021401+0000","scrub_duration":0.035349778999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6514","reported_seq":6542,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:56.306380+0000","last_change":"2024-05-08T23:41:56.306531+0000","last_active":"2024-05-08T23:41:56.306380+0000","last_peered":"2024-05-08T23:41:56.306380+0000","last_clean":"2024-05-08T23:41:56.306380+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:41:56.306380+0000","last_undegraded":"2024-05-08T23:41:56.306380+0000","last_fullsized":"2024-05-08T23:41:56.306380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6514","last_scrub_stamp":"2024-05-08T23:41:56.306349+0000","last_deep_scrub":"23'6514","last_deep_scrub_stamp":"2024-05-08T23:41:56.306349+0000","last_clean_scrub_stamp":"2024-05-08T23:41:56.306349+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:43.050878+0000","scrub_duration":0.023713415000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5995","reported_seq":6023,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:52.038490+0000","last_change":"2024-05-08T23:41:52.038637+0000","last_active":"2024-05-08T23:41:52.038490+0000","last_peered":"2024-05-08T23:41:52.038490+0000","last_clean":"2024-05-08T23:41:52.038490+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:41:52.038490+0000","last_undegraded":"2024-05-08T23:41:52.038490+0000","last_fullsized":"2024-05-08T23:41:52.038490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5995","last_scrub_stamp":"2024-05-08T23:41:52.038462+0000","last_deep_scrub":"23'5995","last_deep_scrub_stamp":"2024-05-08T23:41:52.038462+0000","last_clean_scrub_stamp":"2024-05-08T23:41:52.038462+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:21.325377+0000","scrub_duration":0.035174748999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7607","reported_seq":7632,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:53.371310+0000","last_change":"2024-05-08T23:41:53.371457+0000","last_active":"2024-05-08T23:41:53.371310+0000","last_peered":"2024-05-08T23:41:53.371310+0000","last_clean":"2024-05-08T23:41:53.371310+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:41:53.371310+0000","last_undegraded":"2024-05-08T23:41:53.371310+0000","last_fullsized":"2024-05-08T23:41:53.371310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7607","last_scrub_stamp":"2024-05-08T23:41:53.371280+0000","last_deep_scrub":"23'7607","last_deep_scrub_stamp":"2024-05-08T23:41:53.371280+0000","last_clean_scrub_stamp":"2024-05-08T23:41:53.371280+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:11.346605+0000","scrub_duration":0.019457261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5753","reported_seq":5780,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:58.091917+0000","last_change":"2024-05-08T23:41:58.092065+0000","last_active":"2024-05-08T23:41:58.091917+0000","last_peered":"2024-05-08T23:41:58.091917+0000","last_clean":"2024-05-08T23:41:58.091917+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:41:58.091917+0000","last_undegraded":"2024-05-08T23:41:58.091917+0000","last_fullsized":"2024-05-08T23:41:58.091917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5753","last_scrub_stamp":"2024-05-08T23:41:58.091887+0000","last_deep_scrub":"23'5753","last_deep_scrub_stamp":"2024-05-08T23:41:58.091887+0000","last_clean_scrub_stamp":"2024-05-08T23:41:58.091887+0000","objects_scrubbed":0,"log_size":5753,"log_dups_size":0,"ondisk_log_size":5753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:10.243950+0000","scrub_duration":0.030293665000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6497","reported_seq":6525,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:48.826398+0000","last_change":"2024-05-08T23:41:48.826538+0000","last_active":"2024-05-08T23:41:48.826398+0000","last_peered":"2024-05-08T23:41:48.826398+0000","last_clean":"2024-05-08T23:41:48.826398+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:41:48.826398+0000","last_undegraded":"2024-05-08T23:41:48.826398+0000","last_fullsized":"2024-05-08T23:41:48.826398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6497","last_scrub_stamp":"2024-05-08T23:41:48.826374+0000","last_deep_scrub":"23'6497","last_deep_scrub_stamp":"2024-05-08T23:41:48.826374+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.826374+0000","objects_scrubbed":0,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:29.649909+0000","scrub_duration":0.026040191000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6893","reported_seq":6920,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.834294+0000","last_change":"2024-05-08T23:41:45.834441+0000","last_active":"2024-05-08T23:41:45.834294+0000","last_peered":"2024-05-08T23:41:45.834294+0000","last_clean":"2024-05-08T23:41:45.834294+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:41:45.834294+0000","last_undegraded":"2024-05-08T23:41:45.834294+0000","last_fullsized":"2024-05-08T23:41:45.834294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6893","last_scrub_stamp":"2024-05-08T23:41:45.834264+0000","last_deep_scrub":"23'6893","last_deep_scrub_stamp":"2024-05-08T23:41:45.834264+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.834264+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:19.348809+0000","scrub_duration":0.031566932999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8092","reported_seq":8119,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:54.863870+0000","last_change":"2024-05-08T23:41:54.864020+0000","last_active":"2024-05-08T23:41:54.863870+0000","last_peered":"2024-05-08T23:41:54.863870+0000","last_clean":"2024-05-08T23:41:54.863870+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:41:54.863870+0000","last_undegraded":"2024-05-08T23:41:54.863870+0000","last_fullsized":"2024-05-08T23:41:54.863870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8092","last_scrub_stamp":"2024-05-08T23:41:54.863843+0000","last_deep_scrub":"23'8092","last_deep_scrub_stamp":"2024-05-08T23:41:54.863843+0000","last_clean_scrub_stamp":"2024-05-08T23:41:54.863843+0000","objects_scrubbed":0,"log_size":8092,"log_dups_size":0,"ondisk_log_size":8092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:22:09.761580+0000","scrub_duration":0.029109843999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7426","reported_seq":7453,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.227352+0000","last_change":"2024-05-08T23:41:47.227494+0000","last_active":"2024-05-08T23:41:47.227352+0000","last_peered":"2024-05-08T23:41:47.227352+0000","last_clean":"2024-05-08T23:41:47.227352+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:41:47.227352+0000","last_undegraded":"2024-05-08T23:41:47.227352+0000","last_fullsized":"2024-05-08T23:41:47.227352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7426","last_scrub_stamp":"2024-05-08T23:41:47.227326+0000","last_deep_scrub":"23'7426","last_deep_scrub_stamp":"2024-05-08T23:41:47.227326+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.227326+0000","objects_scrubbed":0,"log_size":7426,"log_dups_size":0,"ondisk_log_size":7426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:10.251198+0000","scrub_duration":0.035983633000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6322","reported_seq":6350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.199366+0000","last_change":"2024-05-08T23:41:45.199366+0000","last_active":"2024-05-08T23:41:45.199366+0000","last_peered":"2024-05-08T23:41:45.199366+0000","last_clean":"2024-05-08T23:41:45.199366+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:41:45.199366+0000","last_undegraded":"2024-05-08T23:41:45.199366+0000","last_fullsized":"2024-05-08T23:41:45.199366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6322","last_scrub_stamp":"2024-05-08T23:41:45.199187+0000","last_deep_scrub":"23'6322","last_deep_scrub_stamp":"2024-05-08T23:41:45.199187+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.199187+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:33.273078+0000","scrub_duration":0.025082496999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5509","reported_seq":5537,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.529218+0000","last_change":"2024-05-08T23:41:29.529218+0000","last_active":"2024-05-08T23:41:29.529218+0000","last_peered":"2024-05-08T23:41:29.529218+0000","last_clean":"2024-05-08T23:41:29.529218+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:41:29.529218+0000","last_undegraded":"2024-05-08T23:41:29.529218+0000","last_fullsized":"2024-05-08T23:41:29.529218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5509","last_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_deep_scrub":"23'5509","last_deep_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.529050+0000","objects_scrubbed":0,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:10.031582+0000","scrub_duration":0.044696347999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6039","reported_seq":6067,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.842084+0000","last_change":"2024-05-08T23:41:44.842084+0000","last_active":"2024-05-08T23:41:44.842084+0000","last_peered":"2024-05-08T23:41:44.842084+0000","last_clean":"2024-05-08T23:41:44.842084+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:41:44.842084+0000","last_undegraded":"2024-05-08T23:41:44.842084+0000","last_fullsized":"2024-05-08T23:41:44.842084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6039","last_scrub_stamp":"2024-05-08T23:41:44.841908+0000","last_deep_scrub":"23'6039","last_deep_scrub_stamp":"2024-05-08T23:41:44.841908+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.841908+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:09.449329+0000","scrub_duration":0.02383439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6680","reported_seq":6697,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:15.549229+0000","last_change":"2024-05-08T23:32:10.348020+0000","last_active":"2024-05-08T23:41:15.549229+0000","last_peered":"2024-05-08T23:41:15.549229+0000","last_clean":"2024-05-08T23:41:15.549229+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:41:15.549229+0000","last_undegraded":"2024-05-08T23:41:15.549229+0000","last_fullsized":"2024-05-08T23:41:15.549229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":6680,"log_dups_size":0,"ondisk_log_size":6680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6521","reported_seq":6549,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.291381+0000","last_change":"2024-05-08T23:41:44.291533+0000","last_active":"2024-05-08T23:41:44.291381+0000","last_peered":"2024-05-08T23:41:44.291381+0000","last_clean":"2024-05-08T23:41:44.291381+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:41:44.291381+0000","last_undegraded":"2024-05-08T23:41:44.291381+0000","last_fullsized":"2024-05-08T23:41:44.291381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6521","last_scrub_stamp":"2024-05-08T23:41:44.291355+0000","last_deep_scrub":"23'6521","last_deep_scrub_stamp":"2024-05-08T23:41:44.291355+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.291355+0000","objects_scrubbed":0,"log_size":6521,"log_dups_size":0,"ondisk_log_size":6521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:50.345328+0000","scrub_duration":0.031414708,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5781","reported_seq":5806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:33.687352+0000","last_change":"2024-05-08T23:41:33.687507+0000","last_active":"2024-05-08T23:41:33.687352+0000","last_peered":"2024-05-08T23:41:33.687352+0000","last_clean":"2024-05-08T23:41:33.687352+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:41:33.687352+0000","last_undegraded":"2024-05-08T23:41:33.687352+0000","last_fullsized":"2024-05-08T23:41:33.687352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5781","last_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_deep_scrub":"23'5781","last_deep_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_clean_scrub_stamp":"2024-05-08T23:41:33.687326+0000","objects_scrubbed":0,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:58.223805+0000","scrub_duration":0.028884346000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7981","reported_seq":8009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.740206+0000","last_change":"2024-05-08T23:41:28.740206+0000","last_active":"2024-05-08T23:41:28.740206+0000","last_peered":"2024-05-08T23:41:28.740206+0000","last_clean":"2024-05-08T23:41:28.740206+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:41:28.740206+0000","last_undegraded":"2024-05-08T23:41:28.740206+0000","last_fullsized":"2024-05-08T23:41:28.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7981","last_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_deep_scrub":"23'7981","last_deep_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.740034+0000","objects_scrubbed":0,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:13.685241+0000","scrub_duration":0.017426704000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5310","reported_seq":5338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.620114+0000","last_change":"2024-05-08T23:41:28.620267+0000","last_active":"2024-05-08T23:41:28.620114+0000","last_peered":"2024-05-08T23:41:28.620114+0000","last_clean":"2024-05-08T23:41:28.620114+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:41:28.620114+0000","last_undegraded":"2024-05-08T23:41:28.620114+0000","last_fullsized":"2024-05-08T23:41:28.620114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5310","last_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_deep_scrub":"23'5310","last_deep_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.620084+0000","objects_scrubbed":0,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:05.218724+0000","scrub_duration":0.024435924000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6330","reported_seq":6358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:57.848646+0000","last_change":"2024-05-08T23:41:57.848795+0000","last_active":"2024-05-08T23:41:57.848646+0000","last_peered":"2024-05-08T23:41:57.848646+0000","last_clean":"2024-05-08T23:41:57.848646+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:41:57.848646+0000","last_undegraded":"2024-05-08T23:41:57.848646+0000","last_fullsized":"2024-05-08T23:41:57.848646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6330","last_scrub_stamp":"2024-05-08T23:41:57.848610+0000","last_deep_scrub":"23'6330","last_deep_scrub_stamp":"2024-05-08T23:41:57.848610+0000","last_clean_scrub_stamp":"2024-05-08T23:41:57.848610+0000","objects_scrubbed":0,"log_size":6330,"log_dups_size":0,"ondisk_log_size":6330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:50.731423+0000","scrub_duration":0.027173037000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'386","reported_seq":415,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.402088+0000","last_change":"2024-05-08T23:41:22.402198+0000","last_active":"2024-05-08T23:41:22.402088+0000","last_peered":"2024-05-08T23:41:22.402088+0000","last_clean":"2024-05-08T23:41:22.402088+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:41:22.402088+0000","last_undegraded":"2024-05-08T23:41:22.402088+0000","last_fullsized":"2024-05-08T23:41:22.402088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'386","last_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_deep_scrub":"23'386","last_deep_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.402063+0000","objects_scrubbed":3,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:40.356038+0000","scrub_duration":0.050272126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6847","reported_seq":6873,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.635082+0000","last_change":"2024-05-08T23:41:30.635228+0000","last_active":"2024-05-08T23:41:30.635082+0000","last_peered":"2024-05-08T23:41:30.635082+0000","last_clean":"2024-05-08T23:41:30.635082+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:41:30.635082+0000","last_undegraded":"2024-05-08T23:41:30.635082+0000","last_fullsized":"2024-05-08T23:41:30.635082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6847","last_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_deep_scrub":"23'6847","last_deep_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.635055+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:39.371481+0000","scrub_duration":0.022460726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.929415+0000","last_change":"2024-05-08T23:41:18.929514+0000","last_active":"2024-05-08T23:41:18.929415+0000","last_peered":"2024-05-08T23:41:18.929415+0000","last_clean":"2024-05-08T23:41:18.929415+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:41:18.929415+0000","last_undegraded":"2024-05-08T23:41:18.929415+0000","last_fullsized":"2024-05-08T23:41:18.929415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.929388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.117796+0000","scrub_duration":0.0057181410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7917","reported_seq":7953,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.281894+0000","last_change":"2024-05-08T23:41:49.282025+0000","last_active":"2024-05-08T23:41:49.281894+0000","last_peered":"2024-05-08T23:41:49.281894+0000","last_clean":"2024-05-08T23:41:49.281894+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:41:49.281894+0000","last_undegraded":"2024-05-08T23:41:49.281894+0000","last_fullsized":"2024-05-08T23:41:49.281894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7917","last_scrub_stamp":"2024-05-08T23:41:49.281860+0000","last_deep_scrub":"23'7917","last_deep_scrub_stamp":"2024-05-08T23:41:49.281860+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.281860+0000","objects_scrubbed":0,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.998917+0000","scrub_duration":0.010578298,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'11","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:38.830241+0000","last_change":"2024-05-08T23:41:38.830351+0000","last_active":"2024-05-08T23:41:38.830241+0000","last_peered":"2024-05-08T23:41:38.830241+0000","last_clean":"2024-05-08T23:41:38.830241+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:41:38.830241+0000","last_undegraded":"2024-05-08T23:41:38.830241+0000","last_fullsized":"2024-05-08T23:41:38.830241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-05-08T23:41:38.830214+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-05-08T23:41:38.830214+0000","last_clean_scrub_stamp":"2024-05-08T23:41:38.830214+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:30.484843+0000","scrub_duration":0.003075918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4528","reported_seq":4555,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:46.043553+0000","last_change":"2024-05-08T23:41:46.043684+0000","last_active":"2024-05-08T23:41:46.043553+0000","last_peered":"2024-05-08T23:41:46.043553+0000","last_clean":"2024-05-08T23:41:46.043553+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:41:46.043553+0000","last_undegraded":"2024-05-08T23:41:46.043553+0000","last_fullsized":"2024-05-08T23:41:46.043553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4528","last_scrub_stamp":"2024-05-08T23:41:46.043526+0000","last_deep_scrub":"23'4528","last_deep_scrub_stamp":"2024-05-08T23:41:46.043526+0000","last_clean_scrub_stamp":"2024-05-08T23:41:46.043526+0000","objects_scrubbed":0,"log_size":4528,"log_dups_size":0,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:11.152260+0000","scrub_duration":0.028508444000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'7","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.806630+0000","last_change":"2024-05-08T23:41:30.806782+0000","last_active":"2024-05-08T23:41:30.806630+0000","last_peered":"2024-05-08T23:41:30.806630+0000","last_clean":"2024-05-08T23:41:30.806630+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:41:30.806630+0000","last_undegraded":"2024-05-08T23:41:30.806630+0000","last_fullsized":"2024-05-08T23:41:30.806630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.806600+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:00.816663+0000","scrub_duration":0.0035991980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7912","reported_seq":7939,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.052599+0000","last_change":"2024-05-08T23:41:49.052735+0000","last_active":"2024-05-08T23:41:49.052599+0000","last_peered":"2024-05-08T23:41:49.052599+0000","last_clean":"2024-05-08T23:41:49.052599+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:41:49.052599+0000","last_undegraded":"2024-05-08T23:41:49.052599+0000","last_fullsized":"2024-05-08T23:41:49.052599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7912","last_scrub_stamp":"2024-05-08T23:41:49.052572+0000","last_deep_scrub":"23'7912","last_deep_scrub_stamp":"2024-05-08T23:41:49.052572+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.052572+0000","objects_scrubbed":0,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:35.024925+0000","scrub_duration":0.038943398999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1097","reported_seq":1125,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.879281+0000","last_change":"2024-05-08T23:41:21.879390+0000","last_active":"2024-05-08T23:41:21.879281+0000","last_peered":"2024-05-08T23:41:21.879281+0000","last_clean":"2024-05-08T23:41:21.879281+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:41:21.879281+0000","last_undegraded":"2024-05-08T23:41:21.879281+0000","last_fullsized":"2024-05-08T23:41:21.879281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'1097","last_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_deep_scrub":"23'1097","last_deep_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.879252+0000","objects_scrubbed":1,"log_size":1097,"log_dups_size":0,"ondisk_log_size":1097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:08.291506+0000","scrub_duration":0.026393197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5451","reported_seq":5478,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.439335+0000","last_change":"2024-05-08T23:41:32.439484+0000","last_active":"2024-05-08T23:41:32.439335+0000","last_peered":"2024-05-08T23:41:32.439335+0000","last_clean":"2024-05-08T23:41:32.439335+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:41:32.439335+0000","last_undegraded":"2024-05-08T23:41:32.439335+0000","last_fullsized":"2024-05-08T23:41:32.439335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5451","last_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_deep_scrub":"23'5451","last_deep_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.439309+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:57.926373+0000","scrub_duration":0.032867014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'27","reported_seq":55,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.226600+0000","last_change":"2024-05-08T23:41:23.226727+0000","last_active":"2024-05-08T23:41:23.226600+0000","last_peered":"2024-05-08T23:41:23.226600+0000","last_clean":"2024-05-08T23:41:23.226600+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:41:23.226600+0000","last_undegraded":"2024-05-08T23:41:23.226600+0000","last_fullsized":"2024-05-08T23:41:23.226600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'27","last_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_deep_scrub":"23'27","last_deep_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.226573+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:14.740811+0000","scrub_duration":0.0030473599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5065","reported_seq":5092,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.218177+0000","last_change":"2024-05-08T23:41:30.218312+0000","last_active":"2024-05-08T23:41:30.218177+0000","last_peered":"2024-05-08T23:41:30.218177+0000","last_clean":"2024-05-08T23:41:30.218177+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:41:30.218177+0000","last_undegraded":"2024-05-08T23:41:30.218177+0000","last_fullsized":"2024-05-08T23:41:30.218177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5065","last_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_deep_scrub":"23'5065","last_deep_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.218148+0000","objects_scrubbed":0,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:10.065500+0000","scrub_duration":0.024008964000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.922945+0000","last_change":"2024-05-08T23:41:27.923055+0000","last_active":"2024-05-08T23:41:27.922945+0000","last_peered":"2024-05-08T23:41:27.922945+0000","last_clean":"2024-05-08T23:41:27.922945+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:41:27.922945+0000","last_undegraded":"2024-05-08T23:41:27.922945+0000","last_fullsized":"2024-05-08T23:41:27.922945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.922913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:51.738030+0000","scrub_duration":0.0063254560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6601","reported_seq":6626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.688541+0000","last_change":"2024-05-08T23:41:32.688688+0000","last_active":"2024-05-08T23:41:32.688541+0000","last_peered":"2024-05-08T23:41:32.688541+0000","last_clean":"2024-05-08T23:41:32.688541+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:41:32.688541+0000","last_undegraded":"2024-05-08T23:41:32.688541+0000","last_fullsized":"2024-05-08T23:41:32.688541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6601","last_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_deep_scrub":"23'6601","last_deep_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.688513+0000","objects_scrubbed":0,"log_size":6601,"log_dups_size":0,"ondisk_log_size":6601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:26.541886+0000","scrub_duration":0.031381991999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:26.794830+0000","last_change":"2024-05-08T23:41:26.794830+0000","last_active":"2024-05-08T23:41:26.794830+0000","last_peered":"2024-05-08T23:41:26.794830+0000","last_clean":"2024-05-08T23:41:26.794830+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:41:26.794830+0000","last_undegraded":"2024-05-08T23:41:26.794830+0000","last_fullsized":"2024-05-08T23:41:26.794830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_clean_scrub_stamp":"2024-05-08T23:41:26.794365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:55.135134+0000","scrub_duration":0.001702903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7591","reported_seq":7619,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.322144+0000","last_change":"2024-05-08T23:41:40.322144+0000","last_active":"2024-05-08T23:41:40.322144+0000","last_peered":"2024-05-08T23:41:40.322144+0000","last_clean":"2024-05-08T23:41:40.322144+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:41:40.322144+0000","last_undegraded":"2024-05-08T23:41:40.322144+0000","last_fullsized":"2024-05-08T23:41:40.322144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7591","last_scrub_stamp":"2024-05-08T23:41:40.321987+0000","last_deep_scrub":"23'7591","last_deep_scrub_stamp":"2024-05-08T23:41:40.321987+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.321987+0000","objects_scrubbed":0,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:31.416860+0000","scrub_duration":0.024728031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.154129+0000","last_change":"2024-05-08T23:41:41.154129+0000","last_active":"2024-05-08T23:41:41.154129+0000","last_peered":"2024-05-08T23:41:41.154129+0000","last_clean":"2024-05-08T23:41:41.154129+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:41:41.154129+0000","last_undegraded":"2024-05-08T23:41:41.154129+0000","last_fullsized":"2024-05-08T23:41:41.154129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:41.153968+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:41.153968+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.153968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:17.166397+0000","scrub_duration":0.016023958000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6503","reported_seq":6531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.781900+0000","last_change":"2024-05-08T23:41:29.781900+0000","last_active":"2024-05-08T23:41:29.781900+0000","last_peered":"2024-05-08T23:41:29.781900+0000","last_clean":"2024-05-08T23:41:29.781900+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:41:29.781900+0000","last_undegraded":"2024-05-08T23:41:29.781900+0000","last_fullsized":"2024-05-08T23:41:29.781900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6503","last_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_deep_scrub":"23'6503","last_deep_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.781726+0000","objects_scrubbed":0,"log_size":6503,"log_dups_size":0,"ondisk_log_size":6503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:40:01.229286+0000","scrub_duration":0.043658807000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.653942+0000","last_change":"2024-05-08T23:41:21.653942+0000","last_active":"2024-05-08T23:41:21.653942+0000","last_peered":"2024-05-08T23:41:21.653942+0000","last_clean":"2024-05-08T23:41:21.653942+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:41:21.653942+0000","last_undegraded":"2024-05-08T23:41:21.653942+0000","last_fullsized":"2024-05-08T23:41:21.653942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.653751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:19.453466+0000","scrub_duration":0.0043470959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5821","reported_seq":5857,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:37.117531+0000","last_change":"2024-05-08T23:41:37.117531+0000","last_active":"2024-05-08T23:41:37.117531+0000","last_peered":"2024-05-08T23:41:37.117531+0000","last_clean":"2024-05-08T23:41:37.117531+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:41:37.117531+0000","last_undegraded":"2024-05-08T23:41:37.117531+0000","last_fullsized":"2024-05-08T23:41:37.117531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5821","last_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_deep_scrub":"23'5821","last_deep_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_clean_scrub_stamp":"2024-05-08T23:41:37.117353+0000","objects_scrubbed":0,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:31.821747+0000","scrub_duration":0.021732336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:34.220610+0000","last_change":"2024-05-08T23:41:34.220610+0000","last_active":"2024-05-08T23:41:34.220610+0000","last_peered":"2024-05-08T23:41:34.220610+0000","last_clean":"2024-05-08T23:41:34.220610+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:41:34.220610+0000","last_undegraded":"2024-05-08T23:41:34.220610+0000","last_fullsized":"2024-05-08T23:41:34.220610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_clean_scrub_stamp":"2024-05-08T23:41:34.220429+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:47.942494+0000","scrub_duration":0.017072019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8452","reported_seq":8480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.479224+0000","last_change":"2024-05-08T23:41:42.479224+0000","last_active":"2024-05-08T23:41:42.479224+0000","last_peered":"2024-05-08T23:41:42.479224+0000","last_clean":"2024-05-08T23:41:42.479224+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:41:42.479224+0000","last_undegraded":"2024-05-08T23:41:42.479224+0000","last_fullsized":"2024-05-08T23:41:42.479224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8452","last_scrub_stamp":"2024-05-08T23:41:42.479045+0000","last_deep_scrub":"23'8452","last_deep_scrub_stamp":"2024-05-08T23:41:42.479045+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.479045+0000","objects_scrubbed":0,"log_size":8452,"log_dups_size":0,"ondisk_log_size":8452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:31.371743+0000","scrub_duration":0.025948715000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'226","reported_seq":255,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.134261+0000","last_change":"2024-05-08T23:41:29.134261+0000","last_active":"2024-05-08T23:41:29.134261+0000","last_peered":"2024-05-08T23:41:29.134261+0000","last_clean":"2024-05-08T23:41:29.134261+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:41:29.134261+0000","last_undegraded":"2024-05-08T23:41:29.134261+0000","last_fullsized":"2024-05-08T23:41:29.134261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'226","last_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_deep_scrub":"23'226","last_deep_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.134118+0000","objects_scrubbed":4,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:53.545220+0000","scrub_duration":0.018859172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6156","reported_seq":6184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.797920+0000","last_change":"2024-05-08T23:41:49.798058+0000","last_active":"2024-05-08T23:41:49.797920+0000","last_peered":"2024-05-08T23:41:49.797920+0000","last_clean":"2024-05-08T23:41:49.797920+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:41:49.797920+0000","last_undegraded":"2024-05-08T23:41:49.797920+0000","last_fullsized":"2024-05-08T23:41:49.797920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6156","last_scrub_stamp":"2024-05-08T23:41:49.797892+0000","last_deep_scrub":"23'6156","last_deep_scrub_stamp":"2024-05-08T23:41:49.797892+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.797892+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:32.060564+0000","scrub_duration":0.033242125999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:20.387410+0000","last_change":"2024-05-08T23:41:20.387410+0000","last_active":"2024-05-08T23:41:20.387410+0000","last_peered":"2024-05-08T23:41:20.387410+0000","last_clean":"2024-05-08T23:41:20.387410+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:41:20.387410+0000","last_undegraded":"2024-05-08T23:41:20.387410+0000","last_fullsized":"2024-05-08T23:41:20.387410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_clean_scrub_stamp":"2024-05-08T23:41:20.387268+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:26.731831+0000","scrub_duration":0.0096185279999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5984","reported_seq":6012,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.172489+0000","last_change":"2024-05-08T23:41:39.172630+0000","last_active":"2024-05-08T23:41:39.172489+0000","last_peered":"2024-05-08T23:41:39.172489+0000","last_clean":"2024-05-08T23:41:39.172489+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:41:39.172489+0000","last_undegraded":"2024-05-08T23:41:39.172489+0000","last_fullsized":"2024-05-08T23:41:39.172489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5984","last_scrub_stamp":"2024-05-08T23:41:39.172461+0000","last_deep_scrub":"23'5984","last_deep_scrub_stamp":"2024-05-08T23:41:39.172461+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.172461+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:01.858001+0000","scrub_duration":0.030667745999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.770600+0000","last_change":"2024-05-08T23:41:27.770600+0000","last_active":"2024-05-08T23:41:27.770600+0000","last_peered":"2024-05-08T23:41:27.770600+0000","last_clean":"2024-05-08T23:41:27.770600+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:41:27.770600+0000","last_undegraded":"2024-05-08T23:41:27.770600+0000","last_fullsized":"2024-05-08T23:41:27.770600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.770437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:39.778646+0000","scrub_duration":0.0041469519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7325","reported_seq":7353,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.808810+0000","last_change":"2024-05-08T23:41:47.808947+0000","last_active":"2024-05-08T23:41:47.808810+0000","last_peered":"2024-05-08T23:41:47.808810+0000","last_clean":"2024-05-08T23:41:47.808810+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:41:47.808810+0000","last_undegraded":"2024-05-08T23:41:47.808810+0000","last_fullsized":"2024-05-08T23:41:47.808810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7325","last_scrub_stamp":"2024-05-08T23:41:47.808782+0000","last_deep_scrub":"23'7325","last_deep_scrub_stamp":"2024-05-08T23:41:47.808782+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.808782+0000","objects_scrubbed":0,"log_size":7325,"log_dups_size":0,"ondisk_log_size":7325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:46.865472+0000","scrub_duration":0.029592240999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.183501+0000","last_change":"2024-05-08T23:41:25.183501+0000","last_active":"2024-05-08T23:41:25.183501+0000","last_peered":"2024-05-08T23:41:25.183501+0000","last_clean":"2024-05-08T23:41:25.183501+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:41:25.183501+0000","last_undegraded":"2024-05-08T23:41:25.183501+0000","last_fullsized":"2024-05-08T23:41:25.183501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.183315+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:43.807059+0000","scrub_duration":0.001637573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7017","reported_seq":7045,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.206956+0000","last_change":"2024-05-08T23:41:44.207130+0000","last_active":"2024-05-08T23:41:44.206956+0000","last_peered":"2024-05-08T23:41:44.206956+0000","last_clean":"2024-05-08T23:41:44.206956+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:41:44.206956+0000","last_undegraded":"2024-05-08T23:41:44.206956+0000","last_fullsized":"2024-05-08T23:41:44.206956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7017","last_scrub_stamp":"2024-05-08T23:41:44.206909+0000","last_deep_scrub":"23'7017","last_deep_scrub_stamp":"2024-05-08T23:41:44.206909+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.206909+0000","objects_scrubbed":0,"log_size":7017,"log_dups_size":0,"ondisk_log_size":7017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:41.312327+0000","scrub_duration":0.030209256,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:17.552762+0000","last_change":"2024-05-08T23:41:17.552762+0000","last_active":"2024-05-08T23:41:17.552762+0000","last_peered":"2024-05-08T23:41:17.552762+0000","last_clean":"2024-05-08T23:41:17.552762+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:41:17.552762+0000","last_undegraded":"2024-05-08T23:41:17.552762+0000","last_fullsized":"2024-05-08T23:41:17.552762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_clean_scrub_stamp":"2024-05-08T23:41:17.552644+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:34.431421+0000","scrub_duration":0.0077641810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6611","reported_seq":6639,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.915781+0000","last_change":"2024-05-08T23:41:42.915923+0000","last_active":"2024-05-08T23:41:42.915781+0000","last_peered":"2024-05-08T23:41:42.915781+0000","last_clean":"2024-05-08T23:41:42.915781+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:41:42.915781+0000","last_undegraded":"2024-05-08T23:41:42.915781+0000","last_fullsized":"2024-05-08T23:41:42.915781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6611","last_scrub_stamp":"2024-05-08T23:41:42.915744+0000","last_deep_scrub":"23'6611","last_deep_scrub_stamp":"2024-05-08T23:41:42.915744+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.915744+0000","objects_scrubbed":0,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:30.122347+0000","scrub_duration":0.031668010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.375277+0000","last_change":"2024-05-08T23:41:18.375277+0000","last_active":"2024-05-08T23:41:18.375277+0000","last_peered":"2024-05-08T23:41:18.375277+0000","last_clean":"2024-05-08T23:41:18.375277+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:41:18.375277+0000","last_undegraded":"2024-05-08T23:41:18.375277+0000","last_fullsized":"2024-05-08T23:41:18.375277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.375124+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:49.600961+0000","scrub_duration":0.017542283999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6076","reported_seq":6112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.903972+0000","last_change":"2024-05-08T23:41:41.904122+0000","last_active":"2024-05-08T23:41:41.903972+0000","last_peered":"2024-05-08T23:41:41.903972+0000","last_clean":"2024-05-08T23:41:41.903972+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:41:41.903972+0000","last_undegraded":"2024-05-08T23:41:41.903972+0000","last_fullsized":"2024-05-08T23:41:41.903972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6076","last_scrub_stamp":"2024-05-08T23:41:41.903947+0000","last_deep_scrub":"23'6076","last_deep_scrub_stamp":"2024-05-08T23:41:41.903947+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.903947+0000","objects_scrubbed":0,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:35:21.959582+0000","scrub_duration":0.023963999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.208505+0000","last_change":"2024-05-08T23:41:22.208635+0000","last_active":"2024-05-08T23:41:22.208505+0000","last_peered":"2024-05-08T23:41:22.208505+0000","last_clean":"2024-05-08T23:41:22.208505+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:41:22.208505+0000","last_undegraded":"2024-05-08T23:41:22.208505+0000","last_fullsized":"2024-05-08T23:41:22.208505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.208478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:19.780610+0000","scrub_duration":0.0086337930000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5813","reported_seq":5841,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.912523+0000","last_change":"2024-05-08T23:41:41.912653+0000","last_active":"2024-05-08T23:41:41.912523+0000","last_peered":"2024-05-08T23:41:41.912523+0000","last_clean":"2024-05-08T23:41:41.912523+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:41:41.912523+0000","last_undegraded":"2024-05-08T23:41:41.912523+0000","last_fullsized":"2024-05-08T23:41:41.912523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5813","last_scrub_stamp":"2024-05-08T23:41:41.912496+0000","last_deep_scrub":"23'5813","last_deep_scrub_stamp":"2024-05-08T23:41:41.912496+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.912496+0000","objects_scrubbed":0,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:14.104880+0000","scrub_duration":0.036601382000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.793158+0000","last_change":"2024-05-08T23:41:25.793278+0000","last_active":"2024-05-08T23:41:25.793158+0000","last_peered":"2024-05-08T23:41:25.793158+0000","last_clean":"2024-05-08T23:41:25.793158+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:41:25.793158+0000","last_undegraded":"2024-05-08T23:41:25.793158+0000","last_fullsized":"2024-05-08T23:41:25.793158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.793133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:17.035745+0000","scrub_duration":0.006691158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6336","reported_seq":6364,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.434271+0000","last_change":"2024-05-08T23:41:36.434409+0000","last_active":"2024-05-08T23:41:36.434271+0000","last_peered":"2024-05-08T23:41:36.434271+0000","last_clean":"2024-05-08T23:41:36.434271+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:41:36.434271+0000","last_undegraded":"2024-05-08T23:41:36.434271+0000","last_fullsized":"2024-05-08T23:41:36.434271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6336","last_scrub_stamp":"2024-05-08T23:41:36.434245+0000","last_deep_scrub":"23'6336","last_deep_scrub_stamp":"2024-05-08T23:41:36.434245+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.434245+0000","objects_scrubbed":0,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:06.168585+0000","scrub_duration":0.029133839000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6281","reported_seq":6309,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.366619+0000","last_change":"2024-05-08T23:41:39.366790+0000","last_active":"2024-05-08T23:41:39.366619+0000","last_peered":"2024-05-08T23:41:39.366619+0000","last_clean":"2024-05-08T23:41:39.366619+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:41:39.366619+0000","last_undegraded":"2024-05-08T23:41:39.366619+0000","last_fullsized":"2024-05-08T23:41:39.366619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6281","last_scrub_stamp":"2024-05-08T23:41:39.366592+0000","last_deep_scrub":"23'6281","last_deep_scrub_stamp":"2024-05-08T23:41:39.366592+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.366592+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:07.235788+0000","scrub_duration":0.019830206999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'28","reported_seq":81,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.579643+0000","last_change":"2024-05-08T23:41:25.579753+0000","last_active":"2024-05-08T23:41:25.579643+0000","last_peered":"2024-05-08T23:41:25.579643+0000","last_clean":"2024-05-08T23:41:25.579643+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:41:25.579643+0000","last_undegraded":"2024-05-08T23:41:25.579643+0000","last_fullsized":"2024-05-08T23:41:25.579643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'28","last_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_deep_scrub":"23'28","last_deep_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.579617+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:59.604194+0000","scrub_duration":0.010805047999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6855","reported_seq":6883,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.995720+0000","last_change":"2024-05-08T23:41:28.995863+0000","last_active":"2024-05-08T23:41:28.995720+0000","last_peered":"2024-05-08T23:41:28.995720+0000","last_clean":"2024-05-08T23:41:28.995720+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:41:28.995720+0000","last_undegraded":"2024-05-08T23:41:28.995720+0000","last_fullsized":"2024-05-08T23:41:28.995720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6855","last_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_deep_scrub":"23'6855","last_deep_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.995692+0000","objects_scrubbed":0,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:54.884404+0000","scrub_duration":0.036366062999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.587984+0000","last_change":"2024-05-08T23:41:19.588138+0000","last_active":"2024-05-08T23:41:19.587984+0000","last_peered":"2024-05-08T23:41:19.587984+0000","last_clean":"2024-05-08T23:41:19.587984+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:41:19.587984+0000","last_undegraded":"2024-05-08T23:41:19.587984+0000","last_fullsized":"2024-05-08T23:41:19.587984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.587945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:52.664086+0000","scrub_duration":0.012341421999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.816792+0000","last_change":"2024-05-08T23:41:23.816911+0000","last_active":"2024-05-08T23:41:23.816792+0000","last_peered":"2024-05-08T23:41:23.816792+0000","last_clean":"2024-05-08T23:41:23.816792+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:41:23.816792+0000","last_undegraded":"2024-05-08T23:41:23.816792+0000","last_fullsized":"2024-05-08T23:41:23.816792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.816760+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:47.733691+0000","scrub_duration":0.0062194889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6396","reported_seq":6424,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.256539+0000","last_change":"2024-05-08T23:41:27.256691+0000","last_active":"2024-05-08T23:41:27.256539+0000","last_peered":"2024-05-08T23:41:27.256539+0000","last_clean":"2024-05-08T23:41:27.256539+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:41:27.256539+0000","last_undegraded":"2024-05-08T23:41:27.256539+0000","last_fullsized":"2024-05-08T23:41:27.256539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6396","last_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_deep_scrub":"23'6396","last_deep_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.256513+0000","objects_scrubbed":0,"log_size":6396,"log_dups_size":0,"ondisk_log_size":6396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:10.352345+0000","scrub_duration":0.028481303999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.854006+0000","last_change":"2024-05-08T23:41:24.854120+0000","last_active":"2024-05-08T23:41:24.854006+0000","last_peered":"2024-05-08T23:41:24.854006+0000","last_clean":"2024-05-08T23:41:24.854006+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:41:24.854006+0000","last_undegraded":"2024-05-08T23:41:24.854006+0000","last_fullsized":"2024-05-08T23:41:24.854006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.853976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:46.648846+0000","scrub_duration":0.0020858780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.762775+0000","last_change":"2024-05-08T23:41:28.762886+0000","last_active":"2024-05-08T23:41:28.762775+0000","last_peered":"2024-05-08T23:41:28.762775+0000","last_clean":"2024-05-08T23:41:28.762775+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:41:28.762775+0000","last_undegraded":"2024-05-08T23:41:28.762775+0000","last_fullsized":"2024-05-08T23:41:28.762775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.762740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:30.867016+0000","scrub_duration":0.0025176629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7015","reported_seq":7043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:35.290975+0000","last_change":"2024-05-08T23:41:35.291126+0000","last_active":"2024-05-08T23:41:35.290975+0000","last_peered":"2024-05-08T23:41:35.290975+0000","last_clean":"2024-05-08T23:41:35.290975+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:41:35.290975+0000","last_undegraded":"2024-05-08T23:41:35.290975+0000","last_fullsized":"2024-05-08T23:41:35.290975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7015","last_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_deep_scrub":"23'7015","last_deep_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_clean_scrub_stamp":"2024-05-08T23:41:35.290948+0000","objects_scrubbed":0,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:19.726891+0000","scrub_duration":0.038503554000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'87","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.199741+0000","last_change":"2024-05-08T23:41:18.199836+0000","last_active":"2024-05-08T23:41:18.199741+0000","last_peered":"2024-05-08T23:41:18.199741+0000","last_clean":"2024-05-08T23:41:18.199741+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:41:18.199741+0000","last_undegraded":"2024-05-08T23:41:18.199741+0000","last_fullsized":"2024-05-08T23:41:18.199741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'87","last_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_deep_scrub":"23'87","last_deep_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.199711+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:53.571638+0000","scrub_duration":0.0083734289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6431","reported_seq":6459,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:37.929079+0000","last_change":"2024-05-08T23:41:37.929216+0000","last_active":"2024-05-08T23:41:37.929079+0000","last_peered":"2024-05-08T23:41:37.929079+0000","last_clean":"2024-05-08T23:41:37.929079+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:41:37.929079+0000","last_undegraded":"2024-05-08T23:41:37.929079+0000","last_fullsized":"2024-05-08T23:41:37.929079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6431","last_scrub_stamp":"2024-05-08T23:41:37.929053+0000","last_deep_scrub":"23'6431","last_deep_scrub_stamp":"2024-05-08T23:41:37.929053+0000","last_clean_scrub_stamp":"2024-05-08T23:41:37.929053+0000","objects_scrubbed":0,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:30.984973+0000","scrub_duration":0.027051103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'9","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.238040+0000","last_change":"2024-05-08T23:41:21.238150+0000","last_active":"2024-05-08T23:41:21.238040+0000","last_peered":"2024-05-08T23:41:21.238040+0000","last_clean":"2024-05-08T23:41:21.238040+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:41:21.238040+0000","last_undegraded":"2024-05-08T23:41:21.238040+0000","last_fullsized":"2024-05-08T23:41:21.238040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9","last_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_deep_scrub":"23'9","last_deep_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.238014+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:43.382177+0000","scrub_duration":0.013372182999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5458","reported_seq":5486,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:50.323706+0000","last_change":"2024-05-08T23:41:50.323889+0000","last_active":"2024-05-08T23:41:50.323706+0000","last_peered":"2024-05-08T23:41:50.323706+0000","last_clean":"2024-05-08T23:41:50.323706+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:41:50.323706+0000","last_undegraded":"2024-05-08T23:41:50.323706+0000","last_fullsized":"2024-05-08T23:41:50.323706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5458","last_scrub_stamp":"2024-05-08T23:41:50.323652+0000","last_deep_scrub":"23'5458","last_deep_scrub_stamp":"2024-05-08T23:41:50.323652+0000","last_clean_scrub_stamp":"2024-05-08T23:41:50.323652+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:02.615860+0000","scrub_duration":0.032782893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.864949+0000","last_change":"2024-05-08T23:41:36.865073+0000","last_active":"2024-05-08T23:41:36.864949+0000","last_peered":"2024-05-08T23:41:36.864949+0000","last_clean":"2024-05-08T23:41:36.864949+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:41:36.864949+0000","last_undegraded":"2024-05-08T23:41:36.864949+0000","last_fullsized":"2024-05-08T23:41:36.864949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.864920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:52.653980+0000","scrub_duration":0.0019745399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5844","reported_seq":5872,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.531828+0000","last_change":"2024-05-08T23:41:27.531961+0000","last_active":"2024-05-08T23:41:27.531828+0000","last_peered":"2024-05-08T23:41:27.531828+0000","last_clean":"2024-05-08T23:41:27.531828+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:41:27.531828+0000","last_undegraded":"2024-05-08T23:41:27.531828+0000","last_fullsized":"2024-05-08T23:41:27.531828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5844","last_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_deep_scrub":"23'5844","last_deep_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.531800+0000","objects_scrubbed":0,"log_size":5844,"log_dups_size":0,"ondisk_log_size":5844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:04.924146+0000","scrub_duration":0.023550181999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.185501+0000","last_change":"2024-05-08T23:41:25.185636+0000","last_active":"2024-05-08T23:41:25.185501+0000","last_peered":"2024-05-08T23:41:25.185501+0000","last_clean":"2024-05-08T23:41:25.185501+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:41:25.185501+0000","last_undegraded":"2024-05-08T23:41:25.185501+0000","last_fullsized":"2024-05-08T23:41:25.185501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.185475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:27.661632+0000","scrub_duration":0.009407884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6488","reported_seq":6516,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.822830+0000","last_change":"2024-05-08T23:41:40.822968+0000","last_active":"2024-05-08T23:41:40.822830+0000","last_peered":"2024-05-08T23:41:40.822830+0000","last_clean":"2024-05-08T23:41:40.822830+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:41:40.822830+0000","last_undegraded":"2024-05-08T23:41:40.822830+0000","last_fullsized":"2024-05-08T23:41:40.822830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6488","last_scrub_stamp":"2024-05-08T23:41:40.822802+0000","last_deep_scrub":"23'6488","last_deep_scrub_stamp":"2024-05-08T23:41:40.822802+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.822802+0000","objects_scrubbed":0,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:49.757048+0000","scrub_duration":0.030908683999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:31.230555+0000","last_change":"2024-05-08T23:41:31.230705+0000","last_active":"2024-05-08T23:41:31.230555+0000","last_peered":"2024-05-08T23:41:31.230555+0000","last_clean":"2024-05-08T23:41:31.230555+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:41:31.230555+0000","last_undegraded":"2024-05-08T23:41:31.230555+0000","last_fullsized":"2024-05-08T23:41:31.230555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_clean_scrub_stamp":"2024-05-08T23:41:31.230527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:50.791503+0000","scrub_duration":0.0091231479999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5341","reported_seq":5369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.944446+0000","last_change":"2024-05-08T23:41:30.944603+0000","last_active":"2024-05-08T23:41:30.944446+0000","last_peered":"2024-05-08T23:41:30.944446+0000","last_clean":"2024-05-08T23:41:30.944446+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:41:30.944446+0000","last_undegraded":"2024-05-08T23:41:30.944446+0000","last_fullsized":"2024-05-08T23:41:30.944446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5341","last_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_deep_scrub":"23'5341","last_deep_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.944419+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:37.870106+0000","scrub_duration":0.025456260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'687","reported_seq":716,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.835319+0000","last_change":"2024-05-08T23:41:25.835429+0000","last_active":"2024-05-08T23:41:25.835319+0000","last_peered":"2024-05-08T23:41:25.835319+0000","last_clean":"2024-05-08T23:41:25.835319+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:41:25.835319+0000","last_undegraded":"2024-05-08T23:41:25.835319+0000","last_fullsized":"2024-05-08T23:41:25.835319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'687","last_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_deep_scrub":"23'687","last_deep_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.835291+0000","objects_scrubbed":2,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:44.548476+0000","scrub_duration":0.021024537999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5938","reported_seq":5966,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:53.242233+0000","last_change":"2024-05-08T23:41:53.242369+0000","last_active":"2024-05-08T23:41:53.242233+0000","last_peered":"2024-05-08T23:41:53.242233+0000","last_clean":"2024-05-08T23:41:53.242233+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:41:53.242233+0000","last_undegraded":"2024-05-08T23:41:53.242233+0000","last_fullsized":"2024-05-08T23:41:53.242233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5938","last_scrub_stamp":"2024-05-08T23:41:53.242203+0000","last_deep_scrub":"23'5938","last_deep_scrub_stamp":"2024-05-08T23:41:53.242203+0000","last_clean_scrub_stamp":"2024-05-08T23:41:53.242203+0000","objects_scrubbed":0,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:29.838167+0000","scrub_duration":0.032392742000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.520446+0000","last_change":"2024-05-08T23:41:24.520818+0000","last_active":"2024-05-08T23:41:24.520446+0000","last_peered":"2024-05-08T23:41:24.520446+0000","last_clean":"2024-05-08T23:41:24.520446+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:41:24.520446+0000","last_undegraded":"2024-05-08T23:41:24.520446+0000","last_fullsized":"2024-05-08T23:41:24.520446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.520417+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:30.649499+0000","scrub_duration":0.0021012050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6544","reported_seq":6572,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.866623+0000","last_change":"2024-05-08T23:41:19.866792+0000","last_active":"2024-05-08T23:41:19.866623+0000","last_peered":"2024-05-08T23:41:19.866623+0000","last_clean":"2024-05-08T23:41:19.866623+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:41:19.866623+0000","last_undegraded":"2024-05-08T23:41:19.866623+0000","last_fullsized":"2024-05-08T23:41:19.866623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6544","last_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_deep_scrub":"23'6544","last_deep_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.866594+0000","objects_scrubbed":0,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:33.883813+0000","scrub_duration":0.03064476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.262007+0000","last_change":"2024-05-08T23:41:28.262123+0000","last_active":"2024-05-08T23:41:28.262007+0000","last_peered":"2024-05-08T23:41:28.262007+0000","last_clean":"2024-05-08T23:41:28.262007+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:41:28.262007+0000","last_undegraded":"2024-05-08T23:41:28.262007+0000","last_fullsized":"2024-05-08T23:41:28.262007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.261980+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:56.437680+0000","scrub_duration":0.018641068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6824","reported_seq":6852,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.871008+0000","last_change":"2024-05-08T23:41:36.871129+0000","last_active":"2024-05-08T23:41:36.871008+0000","last_peered":"2024-05-08T23:41:36.871008+0000","last_clean":"2024-05-08T23:41:36.871008+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:41:36.871008+0000","last_undegraded":"2024-05-08T23:41:36.871008+0000","last_fullsized":"2024-05-08T23:41:36.871008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6824","last_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_deep_scrub":"23'6824","last_deep_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.870982+0000","objects_scrubbed":0,"log_size":6824,"log_dups_size":0,"ondisk_log_size":6824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:55.226753+0000","scrub_duration":0.026360792000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.492931+0000","last_change":"2024-05-08T23:41:19.492931+0000","last_active":"2024-05-08T23:41:19.492931+0000","last_peered":"2024-05-08T23:41:19.492931+0000","last_clean":"2024-05-08T23:41:19.492931+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:41:19.492931+0000","last_undegraded":"2024-05-08T23:41:19.492931+0000","last_fullsized":"2024-05-08T23:41:19.492931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.492750+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:25.643838+0000","scrub_duration":0.0040015720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7700","reported_seq":7728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.834733+0000","last_change":"2024-05-08T23:41:44.834881+0000","last_active":"2024-05-08T23:41:44.834733+0000","last_peered":"2024-05-08T23:41:44.834733+0000","last_clean":"2024-05-08T23:41:44.834733+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:41:44.834733+0000","last_undegraded":"2024-05-08T23:41:44.834733+0000","last_fullsized":"2024-05-08T23:41:44.834733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7700","last_scrub_stamp":"2024-05-08T23:41:44.834706+0000","last_deep_scrub":"23'7700","last_deep_scrub_stamp":"2024-05-08T23:41:44.834706+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.834706+0000","objects_scrubbed":0,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:04.647359+0000","scrub_duration":0.028714782000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7806","reported_seq":7834,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:48.199041+0000","last_change":"2024-05-08T23:41:48.199175+0000","last_active":"2024-05-08T23:41:48.199041+0000","last_peered":"2024-05-08T23:41:48.199041+0000","last_clean":"2024-05-08T23:41:48.199041+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:41:48.199041+0000","last_undegraded":"2024-05-08T23:41:48.199041+0000","last_fullsized":"2024-05-08T23:41:48.199041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7806","last_scrub_stamp":"2024-05-08T23:41:48.199014+0000","last_deep_scrub":"23'7806","last_deep_scrub_stamp":"2024-05-08T23:41:48.199014+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.199014+0000","objects_scrubbed":0,"log_size":7806,"log_dups_size":0,"ondisk_log_size":7806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:09.389806+0000","scrub_duration":0.033413879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6574","reported_seq":6602,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.586969+0000","last_change":"2024-05-08T23:41:27.587096+0000","last_active":"2024-05-08T23:41:27.586969+0000","last_peered":"2024-05-08T23:41:27.586969+0000","last_clean":"2024-05-08T23:41:27.586969+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:41:27.586969+0000","last_undegraded":"2024-05-08T23:41:27.586969+0000","last_fullsized":"2024-05-08T23:41:27.586969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6574","last_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_deep_scrub":"23'6574","last_deep_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.586941+0000","objects_scrubbed":0,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:35.348883+0000","scrub_duration":0.027947453000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8085","reported_seq":8113,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.002519+0000","last_change":"2024-05-08T23:41:45.002673+0000","last_active":"2024-05-08T23:41:45.002519+0000","last_peered":"2024-05-08T23:41:45.002519+0000","last_clean":"2024-05-08T23:41:45.002519+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:41:45.002519+0000","last_undegraded":"2024-05-08T23:41:45.002519+0000","last_fullsized":"2024-05-08T23:41:45.002519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8085","last_scrub_stamp":"2024-05-08T23:41:45.002491+0000","last_deep_scrub":"23'8085","last_deep_scrub_stamp":"2024-05-08T23:41:45.002491+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.002491+0000","objects_scrubbed":0,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.725445+0000","scrub_duration":0.037877398,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5657","reported_seq":5685,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:51.330745+0000","last_change":"2024-05-08T23:41:51.330886+0000","last_active":"2024-05-08T23:41:51.330745+0000","last_peered":"2024-05-08T23:41:51.330745+0000","last_clean":"2024-05-08T23:41:51.330745+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:41:51.330745+0000","last_undegraded":"2024-05-08T23:41:51.330745+0000","last_fullsized":"2024-05-08T23:41:51.330745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5657","last_scrub_stamp":"2024-05-08T23:41:51.330715+0000","last_deep_scrub":"23'5657","last_deep_scrub_stamp":"2024-05-08T23:41:51.330715+0000","last_clean_scrub_stamp":"2024-05-08T23:41:51.330715+0000","objects_scrubbed":0,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:52.258234+0000","scrub_duration":0.027254093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6017","reported_seq":6045,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.876533+0000","last_change":"2024-05-08T23:41:42.876674+0000","last_active":"2024-05-08T23:41:42.876533+0000","last_peered":"2024-05-08T23:41:42.876533+0000","last_clean":"2024-05-08T23:41:42.876533+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:41:42.876533+0000","last_undegraded":"2024-05-08T23:41:42.876533+0000","last_fullsized":"2024-05-08T23:41:42.876533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6017","last_scrub_stamp":"2024-05-08T23:41:42.876506+0000","last_deep_scrub":"23'6017","last_deep_scrub_stamp":"2024-05-08T23:41:42.876506+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.876506+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:47.973525+0000","scrub_duration":0.028109640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7053","reported_seq":7081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.364068+0000","last_change":"2024-05-08T23:41:41.364215+0000","last_active":"2024-05-08T23:41:41.364068+0000","last_peered":"2024-05-08T23:41:41.364068+0000","last_clean":"2024-05-08T23:41:41.364068+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:41:41.364068+0000","last_undegraded":"2024-05-08T23:41:41.364068+0000","last_fullsized":"2024-05-08T23:41:41.364068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7053","last_scrub_stamp":"2024-05-08T23:41:41.364041+0000","last_deep_scrub":"23'7053","last_deep_scrub_stamp":"2024-05-08T23:41:41.364041+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.364041+0000","objects_scrubbed":0,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:14.720054+0000","scrub_duration":0.026686147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5666","reported_seq":5683,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:12.152965+0000","last_change":"2024-05-08T23:32:10.355401+0000","last_active":"2024-05-08T23:41:12.152965+0000","last_peered":"2024-05-08T23:41:12.152965+0000","last_clean":"2024-05-08T23:41:12.152965+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:41:12.152965+0000","last_undegraded":"2024-05-08T23:41:12.152965+0000","last_fullsized":"2024-05-08T23:41:12.152965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:32:09.863521+0000","last_clean_scrub_stamp":"2024-05-08T23:32:09.863521+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7698","reported_seq":7726,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.838491+0000","last_change":"2024-05-08T23:41:39.838619+0000","last_active":"2024-05-08T23:41:39.838491+0000","last_peered":"2024-05-08T23:41:39.838491+0000","last_clean":"2024-05-08T23:41:39.838491+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:41:39.838491+0000","last_undegraded":"2024-05-08T23:41:39.838491+0000","last_fullsized":"2024-05-08T23:41:39.838491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7698","last_scrub_stamp":"2024-05-08T23:41:39.838465+0000","last_deep_scrub":"23'7698","last_deep_scrub_stamp":"2024-05-08T23:41:39.838465+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.838465+0000","objects_scrubbed":0,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:07.399697+0000","scrub_duration":0.016337357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5557","reported_seq":5585,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.871363+0000","last_change":"2024-05-08T23:41:40.871507+0000","last_active":"2024-05-08T23:41:40.871363+0000","last_peered":"2024-05-08T23:41:40.871363+0000","last_clean":"2024-05-08T23:41:40.871363+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:41:40.871363+0000","last_undegraded":"2024-05-08T23:41:40.871363+0000","last_fullsized":"2024-05-08T23:41:40.871363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5557","last_scrub_stamp":"2024-05-08T23:41:40.871335+0000","last_deep_scrub":"23'5557","last_deep_scrub_stamp":"2024-05-08T23:41:40.871335+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.871335+0000","objects_scrubbed":0,"log_size":5557,"log_dups_size":0,"ondisk_log_size":5557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:03.187327+0000","scrub_duration":0.026704393999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6393","reported_seq":6421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.942026+0000","last_change":"2024-05-08T23:41:32.942160+0000","last_active":"2024-05-08T23:41:32.942026+0000","last_peered":"2024-05-08T23:41:32.942026+0000","last_clean":"2024-05-08T23:41:32.942026+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:41:32.942026+0000","last_undegraded":"2024-05-08T23:41:32.942026+0000","last_fullsized":"2024-05-08T23:41:32.942026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6393","last_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_deep_scrub":"23'6393","last_deep_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.942001+0000","objects_scrubbed":0,"log_size":6393,"log_dups_size":0,"ondisk_log_size":6393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:28.602761+0000","scrub_duration":0.037578449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7241","reported_seq":7269,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:57.109581+0000","last_change":"2024-05-08T23:41:57.109725+0000","last_active":"2024-05-08T23:41:57.109581+0000","last_peered":"2024-05-08T23:41:57.109581+0000","last_clean":"2024-05-08T23:41:57.109581+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:41:57.109581+0000","last_undegraded":"2024-05-08T23:41:57.109581+0000","last_fullsized":"2024-05-08T23:41:57.109581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7241","last_scrub_stamp":"2024-05-08T23:41:57.109551+0000","last_deep_scrub":"23'7241","last_deep_scrub_stamp":"2024-05-08T23:41:57.109551+0000","last_clean_scrub_stamp":"2024-05-08T23:41:57.109551+0000","objects_scrubbed":0,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:14.284887+0000","scrub_duration":0.037822815000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6810","reported_seq":6838,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:52.329669+0000","last_change":"2024-05-08T23:41:52.329804+0000","last_active":"2024-05-08T23:41:52.329669+0000","last_peered":"2024-05-08T23:41:52.329669+0000","last_clean":"2024-05-08T23:41:52.329669+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:41:52.329669+0000","last_undegraded":"2024-05-08T23:41:52.329669+0000","last_fullsized":"2024-05-08T23:41:52.329669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6810","last_scrub_stamp":"2024-05-08T23:41:52.329638+0000","last_deep_scrub":"23'6810","last_deep_scrub_stamp":"2024-05-08T23:41:52.329638+0000","last_clean_scrub_stamp":"2024-05-08T23:41:52.329638+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:22.093136+0000","scrub_duration":0.029955665999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6394","reported_seq":6422,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:46.180217+0000","last_change":"2024-05-08T23:41:46.180368+0000","last_active":"2024-05-08T23:41:46.180217+0000","last_peered":"2024-05-08T23:41:46.180217+0000","last_clean":"2024-05-08T23:41:46.180217+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:41:46.180217+0000","last_undegraded":"2024-05-08T23:41:46.180217+0000","last_fullsized":"2024-05-08T23:41:46.180217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6394","last_scrub_stamp":"2024-05-08T23:41:46.180190+0000","last_deep_scrub":"23'6394","last_deep_scrub_stamp":"2024-05-08T23:41:46.180190+0000","last_clean_scrub_stamp":"2024-05-08T23:41:46.180190+0000","objects_scrubbed":0,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:39.068684+0000","scrub_duration":0.028385853999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419416,"ondisk_log_size":419416,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293737,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2683,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34587474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2678,"ondisk_log_size":2678,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617404,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":604286,"kb_avail":93754436,"statfs":{"total":96636764160,"available":96004542464,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":618789323},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":8.0020000000000007,"15min":3.2799999999999998},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.9000000000000004,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.441},{"interface":"front","average":{"1min":1.536,"5min":7.2050000000000001,"15min":3.1040000000000001},"min":{"1min":0.34699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":15.487,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.80900000000000005}]},{"osd":1,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":4.0140000000000002,"15min":1.996},"min":{"1min":0.40000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5350000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":7.3179999999999996,"15min":3.161},"min":{"1min":0.36799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":7.9279999999999999,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.81299999999999994}]},{"osd":2,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":3.827,"15min":1.944},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.9589999999999996,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.609,"5min":4.7839999999999998,"15min":2.1850000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.375,"5min":26.084,"15min":26.084},"last":0.73599999999999999}]},{"osd":3,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":6.6429999999999998,"15min":2.9350000000000001},"min":{"1min":0.45300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.2999999999999998,"5min":52.869,"15min":52.869},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.478,"5min":7.1310000000000002,"15min":2.911},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.35499999999999998},"max":{"1min":5.9299999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":4.4610000000000003,"15min":2.089},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.8109999999999999,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.129,"5min":3.5609999999999999,"15min":1.7310000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":7.8550000000000004,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":4.2480000000000002,"15min":2.1600000000000001},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.33100000000000002},"max":{"1min":10.327,"5min":25.116,"15min":63.732999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":6.2969999999999997,"15min":3.028},"min":{"1min":0.46100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.5020000000000007,"5min":37.552999999999997,"15min":62.558},"last":0.84699999999999998}]},{"osd":6,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":2.0070000000000001,"5min":6.8129999999999997,"15min":3.0409999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":13.154999999999999,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.7629999999999999,"5min":4.4379999999999997,"15min":2.056},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.143000000000001,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.68500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4710076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":6.3659999999999997,"15min":2.8140000000000001},"min":{"1min":0.36199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":8.0109999999999992,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":6.9820000000000002,"15min":3.1440000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.25800000000000001},"max":{"1min":3.9660000000000002,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":5.9649999999999999,"15min":2.8879999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4629999999999992,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":5.2009999999999996,"15min":2.6269999999999998},"min":{"1min":0.43099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":4.0460000000000003,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.60499999999999998}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":4.0869999999999997,"15min":2.1139999999999999},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.28599999999999998},"max":{"1min":4.0099999999999998,"5min":41.445,"15min":41.445},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":4.1420000000000003,"15min":2.085},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.0880000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":4.3869999999999996,"15min":2.3119999999999998},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.9060000000000006,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.628},{"interface":"front","average":{"1min":1.2889999999999999,"5min":6.5350000000000001,"15min":3.2010000000000001},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":8.8650000000000002,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":2.9929999999999999,"15min":1.466},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.069,"5min":20.913,"15min":20.913},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.95699999999999996,"5min":3.2480000000000002,"15min":1.831},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.32600000000000001},"max":{"1min":4.1230000000000002,"5min":22.361000000000001,"15min":35.975999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":5.9729999999999999,"15min":2.7229999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.4199999999999999,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":4.3899999999999997,"15min":2.081},"min":{"1min":0.373,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.3130000000000006,"5min":23.105,"15min":23.105},"last":0.41499999999999998}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":3.5179999999999998,"15min":2.1019999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.32300000000000001},"max":{"1min":16.905999999999999,"5min":24.16,"15min":33.463000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.74,"5min":5.5640000000000001,"15min":2.4199999999999999},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.433,"5min":45.805,"15min":45.805},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":5.8040000000000003,"15min":2.5310000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":3.6070000000000002,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":3.8109999999999999,"15min":2.1469999999999998},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":0.98799999999999999,"5min":39.283999999999999,"15min":48.844999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":3.7309999999999999,"15min":1.891},"min":{"1min":0.34200000000000003,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":1.2230000000000001,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.372},{"interface":"front","average":{"1min":0.73699999999999999,"5min":5.2709999999999999,"15min":2.3410000000000002},"min":{"1min":0.44,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":3.9390000000000001,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":6.6470000000000002,"15min":2.972},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.27700000000000002},"max":{"1min":1.1379999999999999,"5min":46.390999999999998,"15min":52.712000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":7.0099999999999998,"15min":3.4780000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32100000000000001},"max":{"1min":1.2929999999999999,"5min":53.265999999999998,"15min":61.832000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":5.3079999999999998,"15min":2.7050000000000001},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":6.8620000000000001,"5min":34.500999999999998,"15min":45.267000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":5.4020000000000001,"15min":2.6099999999999999},"min":{"1min":0.52200000000000002,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":3.7810000000000001,"5min":34.347999999999999,"15min":43.353999999999999},"last":0.72599999999999998}]},{"osd":5,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":5.0700000000000003,"15min":2.6989999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.8200000000000003,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":6.5780000000000003,"15min":3.2839999999999998},"min":{"1min":0.60399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":5.8310000000000004,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":5.6740000000000004,"15min":3.0920000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.9780000000000002,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":5.5620000000000003,"15min":2.9009999999999998},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":6.3860000000000001,"5min":37.802,"15min":42.603999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":7.7400000000000002,"15min":3.4689999999999999},"min":{"1min":0.504,"5min":0.42699999999999999,"15min":0.38100000000000001},"max":{"1min":4.6260000000000003,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":7.6180000000000003,"15min":3.7349999999999999},"min":{"1min":0.52800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.6630000000000003,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":5.5460000000000003,"15min":2.899},"min":{"1min":0.216,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.5140000000000002,"5min":47.241999999999997,"15min":101.899},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":4.5510000000000002,"15min":2.5579999999999998},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.2690000000000001,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.38300000000000001}]},{"osd":2,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":7.4269999999999996,"15min":3.5910000000000002},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.8660000000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":6.5880000000000001,"15min":3.3519999999999999},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":2.3519999999999999,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.423,"5min":6.2320000000000002,"15min":3.383},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":24.239000000000001,"5min":59.796999999999997,"15min":101.929},"last":0.439},{"interface":"front","average":{"1min":2.399,"5min":3.3879999999999999,"15min":2.3410000000000002},"min":{"1min":0.32300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":23.664999999999999,"5min":23.664999999999999,"15min":95.200999999999993},"last":0.40500000000000003}]},{"osd":4,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":6.0060000000000002,"15min":3.2549999999999999},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.35399999999999998},"max":{"1min":6.9850000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.228,"5min":3.7610000000000001,"15min":2.3889999999999998},"min":{"1min":0.46600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.5709999999999997,"5min":19.163,"15min":44.545999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.3410000000000002,"5min":6.1820000000000004,"15min":3.0310000000000001},"min":{"1min":0.44700000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":43.124000000000002,"5min":43.124000000000002,"15min":78.780000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":4.2110000000000003,"5min":5.843,"15min":3.0070000000000001},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":43.109000000000002,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":5.8769999999999998,"15min":3.351},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":11.282999999999999,"5min":45.744999999999997,"15min":101.869},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.284,"5min":5.0599999999999996,"15min":2.944},"min":{"1min":0.48199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":6.8890000000000002,"5min":38.545000000000002,"15min":78.647999999999996},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.4509999999999996,"15min":2.9009999999999998},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":6.9580000000000002,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.482,"5min":3.7200000000000002,"15min":2.3279999999999998},"min":{"1min":0.48999999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.532,"5min":19.065000000000001,"15min":44.713999999999999},"last":0.55900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":5.266,"15min":2.2570000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":9.6880000000000006,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":1.3129999999999999,"5min":4.6859999999999999,"15min":2.149},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.27700000000000002},"max":{"1min":9.7699999999999996,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.39300000000000002}]},{"osd":1,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":5.0739999999999998,"15min":2.5710000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":1.9159999999999999,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.2330000000000001,"5min":5.4859999999999998,"15min":2.524},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.318},"max":{"1min":7.0469999999999997,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.314}]},{"osd":3,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":7.5650000000000004,"15min":3.516},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27600000000000002},"max":{"1min":22.989999999999998,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.371},{"interface":"front","average":{"1min":0.622,"5min":4.0359999999999996,"15min":2.0750000000000002},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":1.6659999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.1959999999999997,"15min":2.363},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":29.513999999999999,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1339999999999999,"5min":3.8719999999999999,"15min":2.0150000000000001},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":7.0970000000000004,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":5.6399999999999997,"15min":2.8500000000000001},"min":{"1min":0.49199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":12.858000000000001,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.853,"5min":5.0960000000000001,"15min":2.371},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":12.723000000000001,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":4.8440000000000003,"15min":2.278},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":10.128,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.647,"5min":5.8680000000000003,"15min":2.605},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":9.3510000000000009,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.56599999999999995}]},{"osd":7,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":4.9199999999999999,"15min":2.3759999999999999},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.5679999999999996,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":2.2879999999999998,"5min":6.5650000000000004,"15min":2.96},"min":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.9250000000000007,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.627}]}]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":5.181,"15min":2.605},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.22600000000000001},"max":{"1min":6.7359999999999998,"5min":53.103999999999999,"15min":67.305999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.5149999999999999,"5min":6.6479999999999997,"15min":3.464},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.592000000000001,"5min":39.784999999999997,"15min":106.934},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":6.1829999999999998,"15min":2.7589999999999999},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":14.182,"5min":76.628,"15min":76.628},"last":0.49099999999999999},{"interface":"front","average":{"1min":2.0030000000000001,"5min":5.4720000000000004,"15min":2.6099999999999999},"min":{"1min":0.39900000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":14.577,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":7.0739999999999998,"15min":3.3050000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":65.519000000000005,"5min":65.519000000000005,"15min":65.519000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.375,"5min":4.75,"15min":2.4169999999999998},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.057,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":7.9180000000000001,"15min":3.5569999999999999},"min":{"1min":0.40799999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":65.575000000000003,"5min":65.575000000000003,"15min":65.575000000000003},"last":0.66400000000000003},{"interface":"front","average":{"1min":3.726,"5min":7.548,"15min":3.137},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":31.77,"5min":68.238,"15min":68.238},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.2990000000000004,"5min":6.6740000000000004,"15min":2.9620000000000002},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.38100000000000001},"max":{"1min":31.806000000000001,"5min":53.381,"15min":53.381},"last":0.71999999999999997},{"interface":"front","average":{"1min":3.157,"5min":6.7670000000000003,"15min":3.294},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":26.997,"5min":32.381,"15min":36.213999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.1429999999999998,"5min":8.0150000000000006,"15min":3.391},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":65.606999999999999,"5min":65.606999999999999,"15min":65.606999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.925,"5min":5.1440000000000001,"15min":2.464},"min":{"1min":0.57099999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":10.94,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.78700000000000003}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9100000000000001,"5min":9.0250000000000004,"15min":3.8580000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":65.638999999999996,"5min":65.638999999999996,"15min":65.638999999999996},"last":0.749},{"interface":"front","average":{"1min":1.421,"5min":6.2400000000000002,"15min":2.9540000000000002},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.681,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":6.1509999999999998,"15min":2.8980000000000001},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.3620000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":2.9729999999999999,"5min":6.7389999999999999,"15min":3.109},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":10.945,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.63300000000000001}]},{"osd":1,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7050000000000001,"5min":5.7030000000000003,"15min":2.9729999999999999},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":24.324999999999999,"5min":35.807000000000002,"15min":40.901000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":5.6130000000000004,"5min":5.867,"15min":2.8839999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":47.481999999999999,"5min":47.481999999999999,"15min":47.481999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.9279999999999999,"5min":5.4900000000000002,"15min":2.871},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":13.356999999999999,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":2.7730000000000001,"5min":4.8339999999999996,"15min":2.5529999999999999},"min":{"1min":0.42199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":11.509,"5min":25.616,"15min":58.899000000000001},"last":0.42799999999999999}]},{"osd":3,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.835,"15min":2.859},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.318},"max":{"1min":24.460999999999999,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":4.3490000000000002,"5min":7.7370000000000001,"15min":3.4350000000000001},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":31.451000000000001,"5min":111.221,"15min":111.221},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":4.6660000000000004,"15min":2.266},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.7229999999999999,"5min":43.497999999999998,"15min":51.871000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.8460000000000001,"5min":4.4720000000000004,"15min":2.0409999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.7100000000000009,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.67800000000000005}]},{"osd":6,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2930000000000001,"5min":4.5890000000000004,"15min":2.0680000000000001},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.436999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":2.2069999999999999,"5min":4.548,"15min":2.3849999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":11.083,"5min":43.774999999999999,"15min":51.932000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":4.0289999999999999,"15min":1.9630000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":7.4790000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":2.6930000000000001,"5min":4.7949999999999999,"15min":2.4729999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":14.221,"5min":44.783000000000001,"15min":51.978000000000002},"last":0.72099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":5.4900000000000002,"15min":2.7029999999999998},"min":{"1min":0.28599999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.4160000000000004,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.8300000000000001,"5min":5.2009999999999996,"15min":2.3119999999999998},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.30199999999999999},"max":{"1min":7.7690000000000001,"5min":44.439,"15min":44.439},"last":0.65500000000000003}]},{"osd":1,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":4.9509999999999996,"15min":2.9039999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":7.2759999999999998,"5min":31.355,"15min":124.85599999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.347,"5min":4.8479999999999999,"15min":2.4580000000000002},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.317},"max":{"1min":7.327,"5min":31.385000000000002,"15min":41.060000000000002},"last":0.64000000000000001}]},{"osd":2,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":6.5110000000000001,"15min":2.9790000000000001},"min":{"1min":0.45600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.8650000000000002,"5min":44.247,"15min":44.247},"last":0.51100000000000001},{"interface":"front","average":{"1min":1.7170000000000001,"5min":4.8680000000000003,"15min":2.3039999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":7.5350000000000001,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":5.0279999999999996,"5min":7.7690000000000001,"15min":3.1699999999999999},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":39.265000000000001,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":4.1520000000000001,"5min":7.9029999999999996,"15min":4.0640000000000001},"min":{"1min":0.46100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.193000000000001,"5min":43.343000000000004,"15min":49.991999999999997},"last":0.60599999999999998}]},{"osd":4,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.1840000000000002,"15min":2.181},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":8.8119999999999994,"5min":43.305,"15min":43.305},"last":0.41199999999999998},{"interface":"front","average":{"1min":1.4199999999999999,"5min":3.669,"15min":1.8049999999999999},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.3780000000000001,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.379}]},{"osd":6,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":5.4550000000000001,"15min":2.855},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":7.8280000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.624},{"interface":"front","average":{"1min":2.0369999999999999,"5min":4.7329999999999997,"15min":2.306},"min":{"1min":0.45000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.2289999999999992,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.498}]},{"osd":7,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":5.6580000000000004,"15min":2.532},"min":{"1min":0.44500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":15.589,"5min":43.262,"15min":43.262},"last":0.55200000000000005},{"interface":"front","average":{"1min":2.2690000000000001,"5min":6.3700000000000001,"15min":3.3159999999999998},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":10.456,"5min":44.018000000000001,"15min":80.302000000000007},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4622048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:42:00.807 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:42:00.809 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-08T23:32:09.863521+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=32, tm_sec=9, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=23, tm_min=41, tm_sec=13, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T23:42:00.809 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T23:42:20.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:42:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.959+0000 7f1a8b380700 1 Processor -- start 2024-05-08T23:42:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.960+0000 7f1a8b380700 1 -- start start 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8b380700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a8415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8b380700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8406e7c0 con 0x7f1a8406ec00 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8b380700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8406e200 con 0x7f1a840b3000 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8b380700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8406f0f0 con 0x7f1a84154280 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8891b700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1a8406ec00 0x7f1a840ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57128/0 (socket says 172.21.15.33:57128) 2024-05-08T23:42:20.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8891b700 1 -- 172.21.15.33:0/990514289 learned_addr learned my addr 172.21.15.33:0/990514289 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a8911c700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a8415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541659820 0 0) 0x7f1a8406e7c0 con 0x7f1a8406ec00 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a700035b0 con 0x7f1a8406ec00 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028012183 0 0) 0x7f1a700035b0 con 0x7f1a8406ec00 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 msgr2=0x7f1a8415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a8415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 >> v1:172.21.15.42:6789/0 conn(0x7f1a840b3000 legacy=0x7f1a84164c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8406e4e0 con 0x7f1a8406ec00 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 151687415 0 0) 0x7f1a80001670 con 0x7f1a8406ec00 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.961+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 >> v1:172.21.15.33:6789/0 conn(0x7f1a8406ec00 legacy=0x7f1a840ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70003a30 0x7f1a70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70004850 0x7f1a70004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a70005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8406f0f0 con 0x7f1a70004850 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000bac0 con 0x7f1a70003a30 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000bc30 con 0x7f1a84154280 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70004850 0x7f1a70004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8911c700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70003a30 0x7f1a70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8991d700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a70005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 -- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 msgr2=0x7f1a70005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a84154280 0x7f1a70005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 -- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70003a30 msgr2=0x7f1a70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70003a30 0x7f1a70003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 -- 172.21.15.33:0/990514289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700035b0 con 0x7f1a70004850 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a8891b700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70004850 0x7f1a70004cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a8406f020 tx=0x7f1a70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f910f065aec80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80007b50 con 0x7f1a70004850 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8000cc70 con 0x7f1a70004850 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.962+0000 7f1a77fff700 1 -- 172.21.15.33:0/990514289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80005960 con 0x7f1a70004850 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 -- 172.21.15.33:0/990514289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70004850 msgr2=0x7f1a70004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 --2- 172.21.15.33:0/990514289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70004850 0x7f1a70004cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a8406f020 tx=0x7f1a70005880 comp rx=0 tx=0).stop 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8991d700 1 -- 172.21.15.33:0/990514289 reap_dead start 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 -- 172.21.15.33:0/990514289 shutdown_connections 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 -- 172.21.15.33:0/990514289 >> 172.21.15.33:0/990514289 conn(0x7f1a84066ae0 msgr2=0x7f1a84066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 -- 172.21.15.33:0/990514289 shutdown_connections 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.963+0000 7f1a8b380700 1 -- 172.21.15.33:0/990514289 wait complete. 2024-05-08T23:42:20.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.964+0000 7f1a8b380700 1 Processor -- start 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 -- start start 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 0x7f1a840caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a840cd770 0x7f1a841576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8415a7c0 con 0x7f1a84154c70 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84155640 con 0x7f1a70004850 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8b380700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840cb570 con 0x7f1a840cd770 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8991d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8991d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38488/0 (socket says 172.21.15.33:38488) 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.965+0000 7f1a8991d700 1 -- 172.21.15.33:0/4080551251 learned_addr learned my addr 172.21.15.33:0/4080551251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8911c700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 0x7f1a840caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8891b700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a840cd770 0x7f1a841576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 -- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a840cd770 msgr2=0x7f1a841576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a840cd770 0x7f1a841576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 -- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 msgr2=0x7f1a840caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 0x7f1a840caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 -- 172.21.15.33:0/4080551251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a841efeb0 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8911c700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 0x7f1a840caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8991d700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a840c8510 tx=0x7f1a7c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c003cd0 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c003e30 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a841f0190 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00dbd0 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.966+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841f0750 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.967+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a84075460 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.968+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f1a7c00dd30 con 0x7f1a84154c70 2024-05-08T23:42:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.968+0000 7f1a75ffb700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 0x7f1a6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.968+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f1a7c0898a0 con 0x7f1a84154c70 2024-05-08T23:42:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.968+0000 7f1a8911c700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 0x7f1a6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.969+0000 7f1a8911c700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 0x7f1a6806d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009c60 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:20.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:20.973+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f1a7c0549b0 con 0x7f1a84154c70 2024-05-08T23:42:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.187+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 --> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a84154660 con 0x7f1a6806ac90 2024-05-08T23:42:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.195+0000 7f1a75ffb700 1 -- 172.21.15.33:0/4080551251 <== mgr.4110 v2:172.21.15.33:6832/46840 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292280 (secure 0 0 0) 0x7f1a78009da0 con 0x7f1a6806ac90 2024-05-08T23:42:21.195 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:42:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.199+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 msgr2=0x7f1a6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.199+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 0x7f1a6806d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009c60 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.199+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 msgr2=0x7f1a84155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1a840c8510 tx=0x7f1a7c00d430 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 shutdown_connections 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f1a6806ac90 0x7f1a6806d150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a84154c70 0x7f1a84155100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a70004850 0x7f1a840caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 --2- 172.21.15.33:0/4080551251 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f1a840cd770 0x7f1a841576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 >> 172.21.15.33:0/4080551251 conn(0x7f1a84066ae0 msgr2=0x7f1a841591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 shutdown_connections 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.200+0000 7f1a8b380700 1 -- 172.21.15.33:0/4080551251 wait complete. 2024-05-08T23:42:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T23:42:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-05-08T23:42:19.513182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17884105,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":140,"num_read_kb":201,"num_write":6320951,"num_write_kb":49019247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":422213,"ondisk_log_size":422213,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":4048792,"kb_used_data":36888,"kb_used_omap":12,"kb_used_meta":4011827,"kb_avail":750925928,"statfs":{"total":773094113280,"available":768948150272,"internally_reserved":0,"allocated":37773312,"data_stored":36472434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":4108111416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"3.3f","version":"23'5702","reported_seq":5730,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:58.802399+0000","last_change":"2024-05-08T23:41:58.802542+0000","last_active":"2024-05-08T23:41:58.802399+0000","last_peered":"2024-05-08T23:41:58.802399+0000","last_clean":"2024-05-08T23:41:58.802399+0000","last_became_active":"2024-05-08T23:32:10.355269+0000","last_became_peered":"2024-05-08T23:32:10.355269+0000","last_unstale":"2024-05-08T23:41:58.802399+0000","last_undegraded":"2024-05-08T23:41:58.802399+0000","last_fullsized":"2024-05-08T23:41:58.802399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5702","last_scrub_stamp":"2024-05-08T23:41:58.802371+0000","last_deep_scrub":"23'5702","last_deep_scrub_stamp":"2024-05-08T23:41:58.802371+0000","last_clean_scrub_stamp":"2024-05-08T23:41:58.802371+0000","objects_scrubbed":0,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:45.817840+0000","scrub_duration":0.0094510960000000008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86025,"num_write_kb":670609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'8392","reported_seq":8420,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.339055+0000","last_change":"2024-05-08T23:41:47.339196+0000","last_active":"2024-05-08T23:41:47.339055+0000","last_peered":"2024-05-08T23:41:47.339055+0000","last_clean":"2024-05-08T23:41:47.339055+0000","last_became_active":"2024-05-08T23:32:10.356027+0000","last_became_peered":"2024-05-08T23:32:10.356027+0000","last_unstale":"2024-05-08T23:41:47.339055+0000","last_undegraded":"2024-05-08T23:41:47.339055+0000","last_fullsized":"2024-05-08T23:41:47.339055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8392","last_scrub_stamp":"2024-05-08T23:41:47.339029+0000","last_deep_scrub":"23'8392","last_deep_scrub_stamp":"2024-05-08T23:41:47.339029+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.339029+0000","objects_scrubbed":0,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:35.021401+0000","scrub_duration":0.035349778999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126819,"num_write_kb":977987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'6514","reported_seq":6542,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:56.306380+0000","last_change":"2024-05-08T23:41:56.306531+0000","last_active":"2024-05-08T23:41:56.306380+0000","last_peered":"2024-05-08T23:41:56.306380+0000","last_clean":"2024-05-08T23:41:56.306380+0000","last_became_active":"2024-05-08T23:32:10.349948+0000","last_became_peered":"2024-05-08T23:32:10.349948+0000","last_unstale":"2024-05-08T23:41:56.306380+0000","last_undegraded":"2024-05-08T23:41:56.306380+0000","last_fullsized":"2024-05-08T23:41:56.306380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6514","last_scrub_stamp":"2024-05-08T23:41:56.306349+0000","last_deep_scrub":"23'6514","last_deep_scrub_stamp":"2024-05-08T23:41:56.306349+0000","last_clean_scrub_stamp":"2024-05-08T23:41:56.306349+0000","objects_scrubbed":0,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:25:43.050878+0000","scrub_duration":0.023713415000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97822,"num_write_kb":754936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'5995","reported_seq":6023,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:52.038490+0000","last_change":"2024-05-08T23:41:52.038637+0000","last_active":"2024-05-08T23:41:52.038490+0000","last_peered":"2024-05-08T23:41:52.038490+0000","last_clean":"2024-05-08T23:41:52.038490+0000","last_became_active":"2024-05-08T23:32:10.351165+0000","last_became_peered":"2024-05-08T23:32:10.351165+0000","last_unstale":"2024-05-08T23:41:52.038490+0000","last_undegraded":"2024-05-08T23:41:52.038490+0000","last_fullsized":"2024-05-08T23:41:52.038490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5995","last_scrub_stamp":"2024-05-08T23:41:52.038462+0000","last_deep_scrub":"23'5995","last_deep_scrub_stamp":"2024-05-08T23:41:52.038462+0000","last_clean_scrub_stamp":"2024-05-08T23:41:52.038462+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:21.325377+0000","scrub_duration":0.035174748999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90709,"num_write_kb":708748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'7607","reported_seq":7632,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:53.371310+0000","last_change":"2024-05-08T23:41:53.371457+0000","last_active":"2024-05-08T23:41:53.371310+0000","last_peered":"2024-05-08T23:41:53.371310+0000","last_clean":"2024-05-08T23:41:53.371310+0000","last_became_active":"2024-05-08T23:32:59.855047+0000","last_became_peered":"2024-05-08T23:32:59.855047+0000","last_unstale":"2024-05-08T23:41:53.371310+0000","last_undegraded":"2024-05-08T23:41:53.371310+0000","last_fullsized":"2024-05-08T23:41:53.371310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7607","last_scrub_stamp":"2024-05-08T23:41:53.371280+0000","last_deep_scrub":"23'7607","last_deep_scrub_stamp":"2024-05-08T23:41:53.371280+0000","last_clean_scrub_stamp":"2024-05-08T23:41:53.371280+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:33:11.346605+0000","scrub_duration":0.019457261,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114566,"num_write_kb":879571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"23'5753","reported_seq":5781,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:58.092065+0000","last_change":"2024-05-08T23:41:58.092065+0000","last_active":"2024-05-08T23:41:58.092065+0000","last_peered":"2024-05-08T23:41:58.092065+0000","last_clean":"2024-05-08T23:41:58.092065+0000","last_became_active":"2024-05-08T23:32:10.355604+0000","last_became_peered":"2024-05-08T23:32:10.355604+0000","last_unstale":"2024-05-08T23:41:58.092065+0000","last_undegraded":"2024-05-08T23:41:58.092065+0000","last_fullsized":"2024-05-08T23:41:58.092065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5753","last_scrub_stamp":"2024-05-08T23:41:58.091887+0000","last_deep_scrub":"23'5753","last_deep_scrub_stamp":"2024-05-08T23:41:58.091887+0000","last_clean_scrub_stamp":"2024-05-08T23:41:58.091887+0000","objects_scrubbed":0,"log_size":5753,"log_dups_size":0,"ondisk_log_size":5753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:10.243950+0000","scrub_duration":0.030293665000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86552,"num_write_kb":672834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'6497","reported_seq":6525,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:48.826398+0000","last_change":"2024-05-08T23:41:48.826538+0000","last_active":"2024-05-08T23:41:48.826398+0000","last_peered":"2024-05-08T23:41:48.826398+0000","last_clean":"2024-05-08T23:41:48.826398+0000","last_became_active":"2024-05-08T23:32:10.352729+0000","last_became_peered":"2024-05-08T23:32:10.352729+0000","last_unstale":"2024-05-08T23:41:48.826398+0000","last_undegraded":"2024-05-08T23:41:48.826398+0000","last_fullsized":"2024-05-08T23:41:48.826398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6497","last_scrub_stamp":"2024-05-08T23:41:48.826374+0000","last_deep_scrub":"23'6497","last_deep_scrub_stamp":"2024-05-08T23:41:48.826374+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.826374+0000","objects_scrubbed":0,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:29.649909+0000","scrub_duration":0.026040191000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98106,"num_write_kb":762786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'6893","reported_seq":6921,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.834441+0000","last_change":"2024-05-08T23:41:45.834441+0000","last_active":"2024-05-08T23:41:45.834441+0000","last_peered":"2024-05-08T23:41:45.834441+0000","last_clean":"2024-05-08T23:41:45.834441+0000","last_became_active":"2024-05-08T23:32:10.347973+0000","last_became_peered":"2024-05-08T23:32:10.347973+0000","last_unstale":"2024-05-08T23:41:45.834441+0000","last_undegraded":"2024-05-08T23:41:45.834441+0000","last_fullsized":"2024-05-08T23:41:45.834441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6893","last_scrub_stamp":"2024-05-08T23:41:45.834264+0000","last_deep_scrub":"23'6893","last_deep_scrub_stamp":"2024-05-08T23:41:45.834264+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.834264+0000","objects_scrubbed":0,"log_size":6893,"log_dups_size":0,"ondisk_log_size":6893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:19.348809+0000","scrub_duration":0.031566932999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103931,"num_write_kb":804158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'8092","reported_seq":8120,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:54.864020+0000","last_change":"2024-05-08T23:41:54.864020+0000","last_active":"2024-05-08T23:41:54.864020+0000","last_peered":"2024-05-08T23:41:54.864020+0000","last_clean":"2024-05-08T23:41:54.864020+0000","last_became_active":"2024-05-08T23:32:10.347332+0000","last_became_peered":"2024-05-08T23:32:10.347332+0000","last_unstale":"2024-05-08T23:41:54.864020+0000","last_undegraded":"2024-05-08T23:41:54.864020+0000","last_fullsized":"2024-05-08T23:41:54.864020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8092","last_scrub_stamp":"2024-05-08T23:41:54.863843+0000","last_deep_scrub":"23'8092","last_deep_scrub_stamp":"2024-05-08T23:41:54.863843+0000","last_clean_scrub_stamp":"2024-05-08T23:41:54.863843+0000","objects_scrubbed":0,"log_size":8092,"log_dups_size":0,"ondisk_log_size":8092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:22:09.761580+0000","scrub_duration":0.029109843999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122169,"num_write_kb":953994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'7426","reported_seq":7454,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.227494+0000","last_change":"2024-05-08T23:41:47.227494+0000","last_active":"2024-05-08T23:41:47.227494+0000","last_peered":"2024-05-08T23:41:47.227494+0000","last_clean":"2024-05-08T23:41:47.227494+0000","last_became_active":"2024-05-08T23:32:10.355086+0000","last_became_peered":"2024-05-08T23:32:10.355086+0000","last_unstale":"2024-05-08T23:41:47.227494+0000","last_undegraded":"2024-05-08T23:41:47.227494+0000","last_fullsized":"2024-05-08T23:41:47.227494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7426","last_scrub_stamp":"2024-05-08T23:41:47.227326+0000","last_deep_scrub":"23'7426","last_deep_scrub_stamp":"2024-05-08T23:41:47.227326+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.227326+0000","objects_scrubbed":0,"log_size":7426,"log_dups_size":0,"ondisk_log_size":7426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:10.251198+0000","scrub_duration":0.035983633000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112086,"num_write_kb":870913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'6322","reported_seq":6350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.199366+0000","last_change":"2024-05-08T23:41:45.199366+0000","last_active":"2024-05-08T23:41:45.199366+0000","last_peered":"2024-05-08T23:41:45.199366+0000","last_clean":"2024-05-08T23:41:45.199366+0000","last_became_active":"2024-05-08T23:32:10.354638+0000","last_became_peered":"2024-05-08T23:32:10.354638+0000","last_unstale":"2024-05-08T23:41:45.199366+0000","last_undegraded":"2024-05-08T23:41:45.199366+0000","last_fullsized":"2024-05-08T23:41:45.199366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6322","last_scrub_stamp":"2024-05-08T23:41:45.199187+0000","last_deep_scrub":"23'6322","last_deep_scrub_stamp":"2024-05-08T23:41:45.199187+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.199187+0000","objects_scrubbed":0,"log_size":6322,"log_dups_size":0,"ondisk_log_size":6322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:33.273078+0000","scrub_duration":0.025082496999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95186,"num_write_kb":733229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'5509","reported_seq":5537,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.529218+0000","last_change":"2024-05-08T23:41:29.529218+0000","last_active":"2024-05-08T23:41:29.529218+0000","last_peered":"2024-05-08T23:41:29.529218+0000","last_clean":"2024-05-08T23:41:29.529218+0000","last_became_active":"2024-05-08T23:32:10.351975+0000","last_became_peered":"2024-05-08T23:32:10.351975+0000","last_unstale":"2024-05-08T23:41:29.529218+0000","last_undegraded":"2024-05-08T23:41:29.529218+0000","last_fullsized":"2024-05-08T23:41:29.529218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5509","last_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_deep_scrub":"23'5509","last_deep_scrub_stamp":"2024-05-08T23:41:29.529050+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.529050+0000","objects_scrubbed":0,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:10.031582+0000","scrub_duration":0.044696347999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82845,"num_write_kb":639156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'6039","reported_seq":6067,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.842084+0000","last_change":"2024-05-08T23:41:44.842084+0000","last_active":"2024-05-08T23:41:44.842084+0000","last_peered":"2024-05-08T23:41:44.842084+0000","last_clean":"2024-05-08T23:41:44.842084+0000","last_became_active":"2024-05-08T23:32:10.351580+0000","last_became_peered":"2024-05-08T23:32:10.351580+0000","last_unstale":"2024-05-08T23:41:44.842084+0000","last_undegraded":"2024-05-08T23:41:44.842084+0000","last_fullsized":"2024-05-08T23:41:44.842084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6039","last_scrub_stamp":"2024-05-08T23:41:44.841908+0000","last_deep_scrub":"23'6039","last_deep_scrub_stamp":"2024-05-08T23:41:44.841908+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.841908+0000","objects_scrubbed":0,"log_size":6039,"log_dups_size":0,"ondisk_log_size":6039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:09.449329+0000","scrub_duration":0.02383439,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90759,"num_write_kb":706229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'6680","reported_seq":6708,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:42:00.237204+0000","last_change":"2024-05-08T23:42:00.237337+0000","last_active":"2024-05-08T23:42:00.237204+0000","last_peered":"2024-05-08T23:42:00.237204+0000","last_clean":"2024-05-08T23:42:00.237204+0000","last_became_active":"2024-05-08T23:32:10.347645+0000","last_became_peered":"2024-05-08T23:32:10.347645+0000","last_unstale":"2024-05-08T23:42:00.237204+0000","last_undegraded":"2024-05-08T23:42:00.237204+0000","last_fullsized":"2024-05-08T23:42:00.237204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6680","last_scrub_stamp":"2024-05-08T23:42:00.237174+0000","last_deep_scrub":"23'6680","last_deep_scrub_stamp":"2024-05-08T23:42:00.237174+0000","last_clean_scrub_stamp":"2024-05-08T23:42:00.237174+0000","objects_scrubbed":0,"log_size":6680,"log_dups_size":0,"ondisk_log_size":6680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:29.371254+0000","scrub_duration":0.039383372999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100659,"num_write_kb":783720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'6521","reported_seq":6549,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.291381+0000","last_change":"2024-05-08T23:41:44.291533+0000","last_active":"2024-05-08T23:41:44.291381+0000","last_peered":"2024-05-08T23:41:44.291381+0000","last_clean":"2024-05-08T23:41:44.291381+0000","last_became_active":"2024-05-08T23:32:10.356022+0000","last_became_peered":"2024-05-08T23:32:10.356022+0000","last_unstale":"2024-05-08T23:41:44.291381+0000","last_undegraded":"2024-05-08T23:41:44.291381+0000","last_fullsized":"2024-05-08T23:41:44.291381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6521","last_scrub_stamp":"2024-05-08T23:41:44.291355+0000","last_deep_scrub":"23'6521","last_deep_scrub_stamp":"2024-05-08T23:41:44.291355+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.291355+0000","objects_scrubbed":0,"log_size":6521,"log_dups_size":0,"ondisk_log_size":6521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:46:50.345328+0000","scrub_duration":0.031414708,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98171,"num_write_kb":760717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'5781","reported_seq":5806,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:33.687352+0000","last_change":"2024-05-08T23:41:33.687507+0000","last_active":"2024-05-08T23:41:33.687352+0000","last_peered":"2024-05-08T23:41:33.687352+0000","last_clean":"2024-05-08T23:41:33.687352+0000","last_became_active":"2024-05-08T23:32:59.855985+0000","last_became_peered":"2024-05-08T23:32:59.855985+0000","last_unstale":"2024-05-08T23:41:33.687352+0000","last_undegraded":"2024-05-08T23:41:33.687352+0000","last_fullsized":"2024-05-08T23:41:33.687352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5781","last_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_deep_scrub":"23'5781","last_deep_scrub_stamp":"2024-05-08T23:41:33.687326+0000","last_clean_scrub_stamp":"2024-05-08T23:41:33.687326+0000","objects_scrubbed":0,"log_size":5781,"log_dups_size":0,"ondisk_log_size":5781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:58.223805+0000","scrub_duration":0.028884346000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87194,"num_write_kb":683547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'7981","reported_seq":8009,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.740206+0000","last_change":"2024-05-08T23:41:28.740206+0000","last_active":"2024-05-08T23:41:28.740206+0000","last_peered":"2024-05-08T23:41:28.740206+0000","last_clean":"2024-05-08T23:41:28.740206+0000","last_became_active":"2024-05-08T23:32:10.348990+0000","last_became_peered":"2024-05-08T23:32:10.348990+0000","last_unstale":"2024-05-08T23:41:28.740206+0000","last_undegraded":"2024-05-08T23:41:28.740206+0000","last_fullsized":"2024-05-08T23:41:28.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7981","last_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_deep_scrub":"23'7981","last_deep_scrub_stamp":"2024-05-08T23:41:28.740034+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.740034+0000","objects_scrubbed":0,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:13.685241+0000","scrub_duration":0.017426704000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120263,"num_write_kb":933221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'5310","reported_seq":5338,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.620114+0000","last_change":"2024-05-08T23:41:28.620267+0000","last_active":"2024-05-08T23:41:28.620114+0000","last_peered":"2024-05-08T23:41:28.620114+0000","last_clean":"2024-05-08T23:41:28.620114+0000","last_became_active":"2024-05-08T23:32:10.347792+0000","last_became_peered":"2024-05-08T23:32:10.347792+0000","last_unstale":"2024-05-08T23:41:28.620114+0000","last_undegraded":"2024-05-08T23:41:28.620114+0000","last_fullsized":"2024-05-08T23:41:28.620114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5310","last_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_deep_scrub":"23'5310","last_deep_scrub_stamp":"2024-05-08T23:41:28.620084+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.620084+0000","objects_scrubbed":0,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:05.218724+0000","scrub_duration":0.024435924000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79523,"num_write_kb":618490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'6330","reported_seq":6358,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:57.848646+0000","last_change":"2024-05-08T23:41:57.848795+0000","last_active":"2024-05-08T23:41:57.848646+0000","last_peered":"2024-05-08T23:41:57.848646+0000","last_clean":"2024-05-08T23:41:57.848646+0000","last_became_active":"2024-05-08T23:32:10.349244+0000","last_became_peered":"2024-05-08T23:32:10.349244+0000","last_unstale":"2024-05-08T23:41:57.848646+0000","last_undegraded":"2024-05-08T23:41:57.848646+0000","last_fullsized":"2024-05-08T23:41:57.848646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6330","last_scrub_stamp":"2024-05-08T23:41:57.848610+0000","last_deep_scrub":"23'6330","last_deep_scrub_stamp":"2024-05-08T23:41:57.848610+0000","last_clean_scrub_stamp":"2024-05-08T23:41:57.848610+0000","objects_scrubbed":0,"log_size":6330,"log_dups_size":0,"ondisk_log_size":6330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:50.731423+0000","scrub_duration":0.027173037000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95110,"num_write_kb":733863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"23'386","reported_seq":415,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.402088+0000","last_change":"2024-05-08T23:41:22.402198+0000","last_active":"2024-05-08T23:41:22.402088+0000","last_peered":"2024-05-08T23:41:22.402088+0000","last_clean":"2024-05-08T23:41:22.402088+0000","last_became_active":"2024-05-08T23:32:09.891826+0000","last_became_peered":"2024-05-08T23:32:09.891826+0000","last_unstale":"2024-05-08T23:41:22.402088+0000","last_undegraded":"2024-05-08T23:41:22.402088+0000","last_fullsized":"2024-05-08T23:41:22.402088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'386","last_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_deep_scrub":"23'386","last_deep_scrub_stamp":"2024-05-08T23:41:22.402063+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.402063+0000","objects_scrubbed":3,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:40.356038+0000","scrub_duration":0.050272126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":380,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'6847","reported_seq":6873,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.635082+0000","last_change":"2024-05-08T23:41:30.635228+0000","last_active":"2024-05-08T23:41:30.635082+0000","last_peered":"2024-05-08T23:41:30.635082+0000","last_clean":"2024-05-08T23:41:30.635082+0000","last_became_active":"2024-05-08T23:33:00.025989+0000","last_became_peered":"2024-05-08T23:33:00.025989+0000","last_unstale":"2024-05-08T23:41:30.635082+0000","last_undegraded":"2024-05-08T23:41:30.635082+0000","last_fullsized":"2024-05-08T23:41:30.635082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6847","last_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_deep_scrub":"23'6847","last_deep_scrub_stamp":"2024-05-08T23:41:30.635055+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.635055+0000","objects_scrubbed":0,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:39.371481+0000","scrub_duration":0.022460726,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103142,"num_write_kb":801798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.929415+0000","last_change":"2024-05-08T23:41:18.929514+0000","last_active":"2024-05-08T23:41:18.929415+0000","last_peered":"2024-05-08T23:41:18.929415+0000","last_clean":"2024-05-08T23:41:18.929415+0000","last_became_active":"2024-05-08T23:32:09.884232+0000","last_became_peered":"2024-05-08T23:32:09.884232+0000","last_unstale":"2024-05-08T23:41:18.929415+0000","last_undegraded":"2024-05-08T23:41:18.929415+0000","last_fullsized":"2024-05-08T23:41:18.929415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:18.929388+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.929388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:41:49.117796+0000","scrub_duration":0.0057181410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'7917","reported_seq":7953,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.281894+0000","last_change":"2024-05-08T23:41:49.282025+0000","last_active":"2024-05-08T23:41:49.281894+0000","last_peered":"2024-05-08T23:41:49.281894+0000","last_clean":"2024-05-08T23:41:49.281894+0000","last_became_active":"2024-05-08T23:32:59.859067+0000","last_became_peered":"2024-05-08T23:32:59.859067+0000","last_unstale":"2024-05-08T23:41:49.281894+0000","last_undegraded":"2024-05-08T23:41:49.281894+0000","last_fullsized":"2024-05-08T23:41:49.281894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7917","last_scrub_stamp":"2024-05-08T23:41:49.281860+0000","last_deep_scrub":"23'7917","last_deep_scrub_stamp":"2024-05-08T23:41:49.281860+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.281860+0000","objects_scrubbed":0,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:12.998917+0000","scrub_duration":0.010578298,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119524,"num_write_kb":928940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:38.830351+0000","last_change":"2024-05-08T23:41:38.830351+0000","last_active":"2024-05-08T23:41:38.830351+0000","last_peered":"2024-05-08T23:41:38.830351+0000","last_clean":"2024-05-08T23:41:38.830351+0000","last_became_active":"2024-05-08T23:32:09.888946+0000","last_became_peered":"2024-05-08T23:32:09.888946+0000","last_unstale":"2024-05-08T23:41:38.830351+0000","last_undegraded":"2024-05-08T23:41:38.830351+0000","last_fullsized":"2024-05-08T23:41:38.830351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-05-08T23:41:38.830214+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-05-08T23:41:38.830214+0000","last_clean_scrub_stamp":"2024-05-08T23:41:38.830214+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:47:30.484843+0000","scrub_duration":0.003075918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4528","reported_seq":4556,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:46.043684+0000","last_change":"2024-05-08T23:41:46.043684+0000","last_active":"2024-05-08T23:41:46.043684+0000","last_peered":"2024-05-08T23:41:46.043684+0000","last_clean":"2024-05-08T23:41:46.043684+0000","last_became_active":"2024-05-08T23:32:10.353265+0000","last_became_peered":"2024-05-08T23:32:10.353265+0000","last_unstale":"2024-05-08T23:41:46.043684+0000","last_undegraded":"2024-05-08T23:41:46.043684+0000","last_fullsized":"2024-05-08T23:41:46.043684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4528","last_scrub_stamp":"2024-05-08T23:41:46.043526+0000","last_deep_scrub":"23'4528","last_deep_scrub_stamp":"2024-05-08T23:41:46.043526+0000","last_clean_scrub_stamp":"2024-05-08T23:41:46.043526+0000","objects_scrubbed":0,"log_size":4528,"log_dups_size":0,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:11.152260+0000","scrub_duration":0.028508444000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67984,"num_write_kb":530585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.806782+0000","last_change":"2024-05-08T23:41:30.806782+0000","last_active":"2024-05-08T23:41:30.806782+0000","last_peered":"2024-05-08T23:41:30.806782+0000","last_clean":"2024-05-08T23:41:30.806782+0000","last_became_active":"2024-05-08T23:32:09.892963+0000","last_became_peered":"2024-05-08T23:32:09.892963+0000","last_unstale":"2024-05-08T23:41:30.806782+0000","last_undegraded":"2024-05-08T23:41:30.806782+0000","last_fullsized":"2024-05-08T23:41:30.806782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:30.806600+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.806600+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:26:00.816663+0000","scrub_duration":0.0035991980000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"23'7912","reported_seq":7940,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.052735+0000","last_change":"2024-05-08T23:41:49.052735+0000","last_active":"2024-05-08T23:41:49.052735+0000","last_peered":"2024-05-08T23:41:49.052735+0000","last_clean":"2024-05-08T23:41:49.052735+0000","last_became_active":"2024-05-08T23:32:10.353311+0000","last_became_peered":"2024-05-08T23:32:10.353311+0000","last_unstale":"2024-05-08T23:41:49.052735+0000","last_undegraded":"2024-05-08T23:41:49.052735+0000","last_fullsized":"2024-05-08T23:41:49.052735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7912","last_scrub_stamp":"2024-05-08T23:41:49.052572+0000","last_deep_scrub":"23'7912","last_deep_scrub_stamp":"2024-05-08T23:41:49.052572+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.052572+0000","objects_scrubbed":0,"log_size":7912,"log_dups_size":0,"ondisk_log_size":7912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:35:35.024925+0000","scrub_duration":0.038943398999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119051,"num_write_kb":921621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1097","reported_seq":1126,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.879390+0000","last_change":"2024-05-08T23:41:21.879390+0000","last_active":"2024-05-08T23:41:21.879390+0000","last_peered":"2024-05-08T23:41:21.879390+0000","last_clean":"2024-05-08T23:41:21.879390+0000","last_became_active":"2024-05-08T23:32:09.885855+0000","last_became_peered":"2024-05-08T23:32:09.885855+0000","last_unstale":"2024-05-08T23:41:21.879390+0000","last_undegraded":"2024-05-08T23:41:21.879390+0000","last_fullsized":"2024-05-08T23:41:21.879390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'1097","last_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_deep_scrub":"23'1097","last_deep_scrub_stamp":"2024-05-08T23:41:21.879252+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.879252+0000","objects_scrubbed":1,"log_size":1097,"log_dups_size":0,"ondisk_log_size":1097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:08.291506+0000","scrub_duration":0.026393197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":5109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"23'5451","reported_seq":5479,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.439484+0000","last_change":"2024-05-08T23:41:32.439484+0000","last_active":"2024-05-08T23:41:32.439484+0000","last_peered":"2024-05-08T23:41:32.439484+0000","last_clean":"2024-05-08T23:41:32.439484+0000","last_became_active":"2024-05-08T23:32:10.343770+0000","last_became_peered":"2024-05-08T23:32:10.343770+0000","last_unstale":"2024-05-08T23:41:32.439484+0000","last_undegraded":"2024-05-08T23:41:32.439484+0000","last_fullsized":"2024-05-08T23:41:32.439484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5451","last_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_deep_scrub":"23'5451","last_deep_scrub_stamp":"2024-05-08T23:41:32.439309+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.439309+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:57.926373+0000","scrub_duration":0.032867014,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81810,"num_write_kb":632105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'27","reported_seq":56,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.226727+0000","last_change":"2024-05-08T23:41:23.226727+0000","last_active":"2024-05-08T23:41:23.226727+0000","last_peered":"2024-05-08T23:41:23.226727+0000","last_clean":"2024-05-08T23:41:23.226727+0000","last_became_active":"2024-05-08T23:32:09.887213+0000","last_became_peered":"2024-05-08T23:32:09.887213+0000","last_unstale":"2024-05-08T23:41:23.226727+0000","last_undegraded":"2024-05-08T23:41:23.226727+0000","last_fullsized":"2024-05-08T23:41:23.226727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'27","last_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_deep_scrub":"23'27","last_deep_scrub_stamp":"2024-05-08T23:41:23.226573+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.226573+0000","objects_scrubbed":1,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:14.740811+0000","scrub_duration":0.0030473599999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"23'5065","reported_seq":5093,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.218312+0000","last_change":"2024-05-08T23:41:30.218312+0000","last_active":"2024-05-08T23:41:30.218312+0000","last_peered":"2024-05-08T23:41:30.218312+0000","last_clean":"2024-05-08T23:41:30.218312+0000","last_became_active":"2024-05-08T23:32:10.358549+0000","last_became_peered":"2024-05-08T23:32:10.358549+0000","last_unstale":"2024-05-08T23:41:30.218312+0000","last_undegraded":"2024-05-08T23:41:30.218312+0000","last_fullsized":"2024-05-08T23:41:30.218312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5065","last_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_deep_scrub":"23'5065","last_deep_scrub_stamp":"2024-05-08T23:41:30.218148+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.218148+0000","objects_scrubbed":0,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:10.065500+0000","scrub_duration":0.024008964000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76308,"num_write_kb":586062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.923055+0000","last_change":"2024-05-08T23:41:27.923055+0000","last_active":"2024-05-08T23:41:27.923055+0000","last_peered":"2024-05-08T23:41:27.923055+0000","last_clean":"2024-05-08T23:41:27.923055+0000","last_became_active":"2024-05-08T23:32:09.885881+0000","last_became_peered":"2024-05-08T23:32:09.885881+0000","last_unstale":"2024-05-08T23:41:27.923055+0000","last_undegraded":"2024-05-08T23:41:27.923055+0000","last_fullsized":"2024-05-08T23:41:27.923055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-05-08T23:41:27.922913+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.922913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:51.738030+0000","scrub_duration":0.0063254560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'6601","reported_seq":6626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.688541+0000","last_change":"2024-05-08T23:41:32.688688+0000","last_active":"2024-05-08T23:41:32.688541+0000","last_peered":"2024-05-08T23:41:32.688541+0000","last_clean":"2024-05-08T23:41:32.688541+0000","last_became_active":"2024-05-08T23:32:59.855356+0000","last_became_peered":"2024-05-08T23:32:59.855356+0000","last_unstale":"2024-05-08T23:41:32.688541+0000","last_undegraded":"2024-05-08T23:41:32.688541+0000","last_fullsized":"2024-05-08T23:41:32.688541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6601","last_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_deep_scrub":"23'6601","last_deep_scrub_stamp":"2024-05-08T23:41:32.688513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.688513+0000","objects_scrubbed":0,"log_size":6601,"log_dups_size":0,"ondisk_log_size":6601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:05:26.541886+0000","scrub_duration":0.031381991999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99307,"num_write_kb":772059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:26.794830+0000","last_change":"2024-05-08T23:41:26.794830+0000","last_active":"2024-05-08T23:41:26.794830+0000","last_peered":"2024-05-08T23:41:26.794830+0000","last_clean":"2024-05-08T23:41:26.794830+0000","last_became_active":"2024-05-08T23:32:09.879585+0000","last_became_peered":"2024-05-08T23:32:09.879585+0000","last_unstale":"2024-05-08T23:41:26.794830+0000","last_undegraded":"2024-05-08T23:41:26.794830+0000","last_fullsized":"2024-05-08T23:41:26.794830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:26.794365+0000","last_clean_scrub_stamp":"2024-05-08T23:41:26.794365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:55.135134+0000","scrub_duration":0.001702903,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'7591","reported_seq":7619,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.322144+0000","last_change":"2024-05-08T23:41:40.322144+0000","last_active":"2024-05-08T23:41:40.322144+0000","last_peered":"2024-05-08T23:41:40.322144+0000","last_clean":"2024-05-08T23:41:40.322144+0000","last_became_active":"2024-05-08T23:32:10.349600+0000","last_became_peered":"2024-05-08T23:32:10.349600+0000","last_unstale":"2024-05-08T23:41:40.322144+0000","last_undegraded":"2024-05-08T23:41:40.322144+0000","last_fullsized":"2024-05-08T23:41:40.322144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7591","last_scrub_stamp":"2024-05-08T23:41:40.321987+0000","last_deep_scrub":"23'7591","last_deep_scrub_stamp":"2024-05-08T23:41:40.321987+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.321987+0000","objects_scrubbed":0,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:31.416860+0000","scrub_duration":0.024728031000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114228,"num_write_kb":883393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.154129+0000","last_change":"2024-05-08T23:41:41.154129+0000","last_active":"2024-05-08T23:41:41.154129+0000","last_peered":"2024-05-08T23:41:41.154129+0000","last_clean":"2024-05-08T23:41:41.154129+0000","last_became_active":"2024-05-08T23:32:09.885044+0000","last_became_peered":"2024-05-08T23:32:09.885044+0000","last_unstale":"2024-05-08T23:41:41.154129+0000","last_undegraded":"2024-05-08T23:41:41.154129+0000","last_fullsized":"2024-05-08T23:41:41.154129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:41.153968+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:41.153968+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.153968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:17.166397+0000","scrub_duration":0.016023958000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'6503","reported_seq":6531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.781900+0000","last_change":"2024-05-08T23:41:29.781900+0000","last_active":"2024-05-08T23:41:29.781900+0000","last_peered":"2024-05-08T23:41:29.781900+0000","last_clean":"2024-05-08T23:41:29.781900+0000","last_became_active":"2024-05-08T23:32:10.349778+0000","last_became_peered":"2024-05-08T23:32:10.349778+0000","last_unstale":"2024-05-08T23:41:29.781900+0000","last_undegraded":"2024-05-08T23:41:29.781900+0000","last_fullsized":"2024-05-08T23:41:29.781900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6503","last_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_deep_scrub":"23'6503","last_deep_scrub_stamp":"2024-05-08T23:41:29.781726+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.781726+0000","objects_scrubbed":0,"log_size":6503,"log_dups_size":0,"ondisk_log_size":6503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:40:01.229286+0000","scrub_duration":0.043658807000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97850,"num_write_kb":750963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.653942+0000","last_change":"2024-05-08T23:41:21.653942+0000","last_active":"2024-05-08T23:41:21.653942+0000","last_peered":"2024-05-08T23:41:21.653942+0000","last_clean":"2024-05-08T23:41:21.653942+0000","last_became_active":"2024-05-08T23:32:09.888551+0000","last_became_peered":"2024-05-08T23:32:09.888551+0000","last_unstale":"2024-05-08T23:41:21.653942+0000","last_undegraded":"2024-05-08T23:41:21.653942+0000","last_fullsized":"2024-05-08T23:41:21.653942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:21.653751+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.653751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:19.453466+0000","scrub_duration":0.0043470959999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'5821","reported_seq":5857,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:37.117531+0000","last_change":"2024-05-08T23:41:37.117531+0000","last_active":"2024-05-08T23:41:37.117531+0000","last_peered":"2024-05-08T23:41:37.117531+0000","last_clean":"2024-05-08T23:41:37.117531+0000","last_became_active":"2024-05-08T23:32:59.859036+0000","last_became_peered":"2024-05-08T23:32:59.859036+0000","last_unstale":"2024-05-08T23:41:37.117531+0000","last_undegraded":"2024-05-08T23:41:37.117531+0000","last_fullsized":"2024-05-08T23:41:37.117531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5821","last_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_deep_scrub":"23'5821","last_deep_scrub_stamp":"2024-05-08T23:41:37.117353+0000","last_clean_scrub_stamp":"2024-05-08T23:41:37.117353+0000","objects_scrubbed":0,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:31.821747+0000","scrub_duration":0.021732336000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87559,"num_write_kb":680480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:34.220610+0000","last_change":"2024-05-08T23:41:34.220610+0000","last_active":"2024-05-08T23:41:34.220610+0000","last_peered":"2024-05-08T23:41:34.220610+0000","last_clean":"2024-05-08T23:41:34.220610+0000","last_became_active":"2024-05-08T23:32:09.884169+0000","last_became_peered":"2024-05-08T23:32:09.884169+0000","last_unstale":"2024-05-08T23:41:34.220610+0000","last_undegraded":"2024-05-08T23:41:34.220610+0000","last_fullsized":"2024-05-08T23:41:34.220610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:34.220429+0000","last_clean_scrub_stamp":"2024-05-08T23:41:34.220429+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:11:47.942494+0000","scrub_duration":0.017072019000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'8452","reported_seq":8480,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.479224+0000","last_change":"2024-05-08T23:41:42.479224+0000","last_active":"2024-05-08T23:41:42.479224+0000","last_peered":"2024-05-08T23:41:42.479224+0000","last_clean":"2024-05-08T23:41:42.479224+0000","last_became_active":"2024-05-08T23:32:10.350334+0000","last_became_peered":"2024-05-08T23:32:10.350334+0000","last_unstale":"2024-05-08T23:41:42.479224+0000","last_undegraded":"2024-05-08T23:41:42.479224+0000","last_fullsized":"2024-05-08T23:41:42.479224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8452","last_scrub_stamp":"2024-05-08T23:41:42.479045+0000","last_deep_scrub":"23'8452","last_deep_scrub_stamp":"2024-05-08T23:41:42.479045+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.479045+0000","objects_scrubbed":0,"log_size":8452,"log_dups_size":0,"ondisk_log_size":8452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:31.371743+0000","scrub_duration":0.025948715000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127249,"num_write_kb":990152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'226","reported_seq":255,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:29.134261+0000","last_change":"2024-05-08T23:41:29.134261+0000","last_active":"2024-05-08T23:41:29.134261+0000","last_peered":"2024-05-08T23:41:29.134261+0000","last_clean":"2024-05-08T23:41:29.134261+0000","last_became_active":"2024-05-08T23:32:09.885267+0000","last_became_peered":"2024-05-08T23:32:09.885267+0000","last_unstale":"2024-05-08T23:41:29.134261+0000","last_undegraded":"2024-05-08T23:41:29.134261+0000","last_fullsized":"2024-05-08T23:41:29.134261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'226","last_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_deep_scrub":"23'226","last_deep_scrub_stamp":"2024-05-08T23:41:29.134118+0000","last_clean_scrub_stamp":"2024-05-08T23:41:29.134118+0000","objects_scrubbed":4,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:53.545220+0000","scrub_duration":0.018859172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'6156","reported_seq":6184,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:49.797920+0000","last_change":"2024-05-08T23:41:49.798058+0000","last_active":"2024-05-08T23:41:49.797920+0000","last_peered":"2024-05-08T23:41:49.797920+0000","last_clean":"2024-05-08T23:41:49.797920+0000","last_became_active":"2024-05-08T23:32:10.360302+0000","last_became_peered":"2024-05-08T23:32:10.360302+0000","last_unstale":"2024-05-08T23:41:49.797920+0000","last_undegraded":"2024-05-08T23:41:49.797920+0000","last_fullsized":"2024-05-08T23:41:49.797920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6156","last_scrub_stamp":"2024-05-08T23:41:49.797892+0000","last_deep_scrub":"23'6156","last_deep_scrub_stamp":"2024-05-08T23:41:49.797892+0000","last_clean_scrub_stamp":"2024-05-08T23:41:49.797892+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:32.060564+0000","scrub_duration":0.033242125999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92782,"num_write_kb":719418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:20.387410+0000","last_change":"2024-05-08T23:41:20.387410+0000","last_active":"2024-05-08T23:41:20.387410+0000","last_peered":"2024-05-08T23:41:20.387410+0000","last_clean":"2024-05-08T23:41:20.387410+0000","last_became_active":"2024-05-08T23:32:09.887338+0000","last_became_peered":"2024-05-08T23:32:09.887338+0000","last_unstale":"2024-05-08T23:41:20.387410+0000","last_undegraded":"2024-05-08T23:41:20.387410+0000","last_fullsized":"2024-05-08T23:41:20.387410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-05-08T23:41:20.387268+0000","last_clean_scrub_stamp":"2024-05-08T23:41:20.387268+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:26.731831+0000","scrub_duration":0.0096185279999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'5984","reported_seq":6012,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.172489+0000","last_change":"2024-05-08T23:41:39.172630+0000","last_active":"2024-05-08T23:41:39.172489+0000","last_peered":"2024-05-08T23:41:39.172489+0000","last_clean":"2024-05-08T23:41:39.172489+0000","last_became_active":"2024-05-08T23:32:10.350495+0000","last_became_peered":"2024-05-08T23:32:10.350495+0000","last_unstale":"2024-05-08T23:41:39.172489+0000","last_undegraded":"2024-05-08T23:41:39.172489+0000","last_fullsized":"2024-05-08T23:41:39.172489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5984","last_scrub_stamp":"2024-05-08T23:41:39.172461+0000","last_deep_scrub":"23'5984","last_deep_scrub_stamp":"2024-05-08T23:41:39.172461+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.172461+0000","objects_scrubbed":0,"log_size":5984,"log_dups_size":0,"ondisk_log_size":5984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:50:01.858001+0000","scrub_duration":0.030667745999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90133,"num_write_kb":703407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.770600+0000","last_change":"2024-05-08T23:41:27.770600+0000","last_active":"2024-05-08T23:41:27.770600+0000","last_peered":"2024-05-08T23:41:27.770600+0000","last_clean":"2024-05-08T23:41:27.770600+0000","last_became_active":"2024-05-08T23:32:09.880910+0000","last_became_peered":"2024-05-08T23:32:09.880910+0000","last_unstale":"2024-05-08T23:41:27.770600+0000","last_undegraded":"2024-05-08T23:41:27.770600+0000","last_fullsized":"2024-05-08T23:41:27.770600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:27.770437+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.770437+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:39.778646+0000","scrub_duration":0.0041469519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'7325","reported_seq":7353,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:47.808810+0000","last_change":"2024-05-08T23:41:47.808947+0000","last_active":"2024-05-08T23:41:47.808810+0000","last_peered":"2024-05-08T23:41:47.808810+0000","last_clean":"2024-05-08T23:41:47.808810+0000","last_became_active":"2024-05-08T23:32:10.358583+0000","last_became_peered":"2024-05-08T23:32:10.358583+0000","last_unstale":"2024-05-08T23:41:47.808810+0000","last_undegraded":"2024-05-08T23:41:47.808810+0000","last_fullsized":"2024-05-08T23:41:47.808810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7325","last_scrub_stamp":"2024-05-08T23:41:47.808782+0000","last_deep_scrub":"23'7325","last_deep_scrub_stamp":"2024-05-08T23:41:47.808782+0000","last_clean_scrub_stamp":"2024-05-08T23:41:47.808782+0000","objects_scrubbed":0,"log_size":7325,"log_dups_size":0,"ondisk_log_size":7325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:46.865472+0000","scrub_duration":0.029592240999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110244,"num_write_kb":856626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.183501+0000","last_change":"2024-05-08T23:41:25.183501+0000","last_active":"2024-05-08T23:41:25.183501+0000","last_peered":"2024-05-08T23:41:25.183501+0000","last_clean":"2024-05-08T23:41:25.183501+0000","last_became_active":"2024-05-08T23:32:09.884990+0000","last_became_peered":"2024-05-08T23:32:09.884990+0000","last_unstale":"2024-05-08T23:41:25.183501+0000","last_undegraded":"2024-05-08T23:41:25.183501+0000","last_fullsized":"2024-05-08T23:41:25.183501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-05-08T23:41:25.183315+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.183315+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:43.807059+0000","scrub_duration":0.001637573,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'7017","reported_seq":7045,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.206956+0000","last_change":"2024-05-08T23:41:44.207130+0000","last_active":"2024-05-08T23:41:44.206956+0000","last_peered":"2024-05-08T23:41:44.206956+0000","last_clean":"2024-05-08T23:41:44.206956+0000","last_became_active":"2024-05-08T23:32:10.351509+0000","last_became_peered":"2024-05-08T23:32:10.351509+0000","last_unstale":"2024-05-08T23:41:44.206956+0000","last_undegraded":"2024-05-08T23:41:44.206956+0000","last_fullsized":"2024-05-08T23:41:44.206956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7017","last_scrub_stamp":"2024-05-08T23:41:44.206909+0000","last_deep_scrub":"23'7017","last_deep_scrub_stamp":"2024-05-08T23:41:44.206909+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.206909+0000","objects_scrubbed":0,"log_size":7017,"log_dups_size":0,"ondisk_log_size":7017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:26:41.312327+0000","scrub_duration":0.030209256,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106032,"num_write_kb":814785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:17.552762+0000","last_change":"2024-05-08T23:41:17.552762+0000","last_active":"2024-05-08T23:41:17.552762+0000","last_peered":"2024-05-08T23:41:17.552762+0000","last_clean":"2024-05-08T23:41:17.552762+0000","last_became_active":"2024-05-08T23:32:09.878237+0000","last_became_peered":"2024-05-08T23:32:09.878237+0000","last_unstale":"2024-05-08T23:41:17.552762+0000","last_undegraded":"2024-05-08T23:41:17.552762+0000","last_fullsized":"2024-05-08T23:41:17.552762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-05-08T23:41:17.552644+0000","last_clean_scrub_stamp":"2024-05-08T23:41:17.552644+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:34.431421+0000","scrub_duration":0.0077641810000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'6611","reported_seq":6639,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.915781+0000","last_change":"2024-05-08T23:41:42.915923+0000","last_active":"2024-05-08T23:41:42.915781+0000","last_peered":"2024-05-08T23:41:42.915781+0000","last_clean":"2024-05-08T23:41:42.915781+0000","last_became_active":"2024-05-08T23:32:10.358647+0000","last_became_peered":"2024-05-08T23:32:10.358647+0000","last_unstale":"2024-05-08T23:41:42.915781+0000","last_undegraded":"2024-05-08T23:41:42.915781+0000","last_fullsized":"2024-05-08T23:41:42.915781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6611","last_scrub_stamp":"2024-05-08T23:41:42.915744+0000","last_deep_scrub":"23'6611","last_deep_scrub_stamp":"2024-05-08T23:41:42.915744+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.915744+0000","objects_scrubbed":0,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:30.122347+0000","scrub_duration":0.031668010000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99655,"num_write_kb":772729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.375277+0000","last_change":"2024-05-08T23:41:18.375277+0000","last_active":"2024-05-08T23:41:18.375277+0000","last_peered":"2024-05-08T23:41:18.375277+0000","last_clean":"2024-05-08T23:41:18.375277+0000","last_became_active":"2024-05-08T23:32:09.893246+0000","last_became_peered":"2024-05-08T23:32:09.893246+0000","last_unstale":"2024-05-08T23:41:18.375277+0000","last_undegraded":"2024-05-08T23:41:18.375277+0000","last_fullsized":"2024-05-08T23:41:18.375277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:18.375124+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.375124+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:49.600961+0000","scrub_duration":0.017542283999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'6076","reported_seq":6112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.903972+0000","last_change":"2024-05-08T23:41:41.904122+0000","last_active":"2024-05-08T23:41:41.903972+0000","last_peered":"2024-05-08T23:41:41.903972+0000","last_clean":"2024-05-08T23:41:41.903972+0000","last_became_active":"2024-05-08T23:33:00.026068+0000","last_became_peered":"2024-05-08T23:33:00.026068+0000","last_unstale":"2024-05-08T23:41:41.903972+0000","last_undegraded":"2024-05-08T23:41:41.903972+0000","last_fullsized":"2024-05-08T23:41:41.903972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6076","last_scrub_stamp":"2024-05-08T23:41:41.903947+0000","last_deep_scrub":"23'6076","last_deep_scrub_stamp":"2024-05-08T23:41:41.903947+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.903947+0000","objects_scrubbed":0,"log_size":6076,"log_dups_size":0,"ondisk_log_size":6076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:35:21.959582+0000","scrub_duration":0.023963999999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91528,"num_write_kb":709069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:22.208505+0000","last_change":"2024-05-08T23:41:22.208635+0000","last_active":"2024-05-08T23:41:22.208505+0000","last_peered":"2024-05-08T23:41:22.208505+0000","last_clean":"2024-05-08T23:41:22.208505+0000","last_became_active":"2024-05-08T23:32:09.898390+0000","last_became_peered":"2024-05-08T23:32:09.898390+0000","last_unstale":"2024-05-08T23:41:22.208505+0000","last_undegraded":"2024-05-08T23:41:22.208505+0000","last_fullsized":"2024-05-08T23:41:22.208505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:22.208478+0000","last_clean_scrub_stamp":"2024-05-08T23:41:22.208478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:41:19.780610+0000","scrub_duration":0.0086337930000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'5813","reported_seq":5841,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.912523+0000","last_change":"2024-05-08T23:41:41.912653+0000","last_active":"2024-05-08T23:41:41.912523+0000","last_peered":"2024-05-08T23:41:41.912523+0000","last_clean":"2024-05-08T23:41:41.912523+0000","last_became_active":"2024-05-08T23:32:10.350739+0000","last_became_peered":"2024-05-08T23:32:10.350739+0000","last_unstale":"2024-05-08T23:41:41.912523+0000","last_undegraded":"2024-05-08T23:41:41.912523+0000","last_fullsized":"2024-05-08T23:41:41.912523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5813","last_scrub_stamp":"2024-05-08T23:41:41.912496+0000","last_deep_scrub":"23'5813","last_deep_scrub_stamp":"2024-05-08T23:41:41.912496+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.912496+0000","objects_scrubbed":0,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:36:14.104880+0000","scrub_duration":0.036601382000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87805,"num_write_kb":684709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.793158+0000","last_change":"2024-05-08T23:41:25.793278+0000","last_active":"2024-05-08T23:41:25.793158+0000","last_peered":"2024-05-08T23:41:25.793158+0000","last_clean":"2024-05-08T23:41:25.793158+0000","last_became_active":"2024-05-08T23:32:09.883616+0000","last_became_peered":"2024-05-08T23:32:09.883616+0000","last_unstale":"2024-05-08T23:41:25.793158+0000","last_undegraded":"2024-05-08T23:41:25.793158+0000","last_fullsized":"2024-05-08T23:41:25.793158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:41:25.793133+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.793133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:17.035745+0000","scrub_duration":0.006691158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6336","reported_seq":6364,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.434271+0000","last_change":"2024-05-08T23:41:36.434409+0000","last_active":"2024-05-08T23:41:36.434271+0000","last_peered":"2024-05-08T23:41:36.434271+0000","last_clean":"2024-05-08T23:41:36.434271+0000","last_became_active":"2024-05-08T23:32:10.350228+0000","last_became_peered":"2024-05-08T23:32:10.350228+0000","last_unstale":"2024-05-08T23:41:36.434271+0000","last_undegraded":"2024-05-08T23:41:36.434271+0000","last_fullsized":"2024-05-08T23:41:36.434271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6336","last_scrub_stamp":"2024-05-08T23:41:36.434245+0000","last_deep_scrub":"23'6336","last_deep_scrub_stamp":"2024-05-08T23:41:36.434245+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.434245+0000","objects_scrubbed":0,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:06.168585+0000","scrub_duration":0.029133839000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95681,"num_write_kb":743606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"23'6281","reported_seq":6309,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.366619+0000","last_change":"2024-05-08T23:41:39.366790+0000","last_active":"2024-05-08T23:41:39.366619+0000","last_peered":"2024-05-08T23:41:39.366619+0000","last_clean":"2024-05-08T23:41:39.366619+0000","last_became_active":"2024-05-08T23:32:10.344499+0000","last_became_peered":"2024-05-08T23:32:10.344499+0000","last_unstale":"2024-05-08T23:41:39.366619+0000","last_undegraded":"2024-05-08T23:41:39.366619+0000","last_fullsized":"2024-05-08T23:41:39.366619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6281","last_scrub_stamp":"2024-05-08T23:41:39.366592+0000","last_deep_scrub":"23'6281","last_deep_scrub_stamp":"2024-05-08T23:41:39.366592+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.366592+0000","objects_scrubbed":0,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:07.235788+0000","scrub_duration":0.019830206999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94594,"num_write_kb":735946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"23'28","reported_seq":81,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.579643+0000","last_change":"2024-05-08T23:41:25.579753+0000","last_active":"2024-05-08T23:41:25.579643+0000","last_peered":"2024-05-08T23:41:25.579643+0000","last_clean":"2024-05-08T23:41:25.579643+0000","last_became_active":"2024-05-08T23:32:09.880094+0000","last_became_peered":"2024-05-08T23:32:09.880094+0000","last_unstale":"2024-05-08T23:41:25.579643+0000","last_undegraded":"2024-05-08T23:41:25.579643+0000","last_fullsized":"2024-05-08T23:41:25.579643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'28","last_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_deep_scrub":"23'28","last_deep_scrub_stamp":"2024-05-08T23:41:25.579617+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.579617+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:59.604194+0000","scrub_duration":0.010805047999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87979,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":24,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"23'6855","reported_seq":6883,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.995720+0000","last_change":"2024-05-08T23:41:28.995863+0000","last_active":"2024-05-08T23:41:28.995720+0000","last_peered":"2024-05-08T23:41:28.995720+0000","last_clean":"2024-05-08T23:41:28.995720+0000","last_became_active":"2024-05-08T23:32:10.352428+0000","last_became_peered":"2024-05-08T23:32:10.352428+0000","last_unstale":"2024-05-08T23:41:28.995720+0000","last_undegraded":"2024-05-08T23:41:28.995720+0000","last_fullsized":"2024-05-08T23:41:28.995720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6855","last_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_deep_scrub":"23'6855","last_deep_scrub_stamp":"2024-05-08T23:41:28.995692+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.995692+0000","objects_scrubbed":0,"log_size":6855,"log_dups_size":0,"ondisk_log_size":6855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:54.884404+0000","scrub_duration":0.036366062999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103369,"num_write_kb":798667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.587984+0000","last_change":"2024-05-08T23:41:19.588138+0000","last_active":"2024-05-08T23:41:19.587984+0000","last_peered":"2024-05-08T23:41:19.587984+0000","last_clean":"2024-05-08T23:41:19.587984+0000","last_became_active":"2024-05-08T23:32:09.887353+0000","last_became_peered":"2024-05-08T23:32:09.887353+0000","last_unstale":"2024-05-08T23:41:19.587984+0000","last_undegraded":"2024-05-08T23:41:19.587984+0000","last_fullsized":"2024-05-08T23:41:19.587984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:19.587945+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.587945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:52.664086+0000","scrub_duration":0.012341421999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'119","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:23.816911+0000","last_change":"2024-05-08T23:41:23.816911+0000","last_active":"2024-05-08T23:41:23.816911+0000","last_peered":"2024-05-08T23:41:23.816911+0000","last_clean":"2024-05-08T23:41:23.816911+0000","last_became_active":"2024-05-08T23:32:07.079079+0000","last_became_peered":"2024-05-08T23:32:07.079079+0000","last_unstale":"2024-05-08T23:41:23.816911+0000","last_undegraded":"2024-05-08T23:41:23.816911+0000","last_fullsized":"2024-05-08T23:41:23.816911+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'119","last_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_deep_scrub":"23'119","last_deep_scrub_stamp":"2024-05-08T23:41:23.816760+0000","last_clean_scrub_stamp":"2024-05-08T23:41:23.816760+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:50:47.733691+0000","scrub_duration":0.0062194889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'6396","reported_seq":6424,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.256539+0000","last_change":"2024-05-08T23:41:27.256691+0000","last_active":"2024-05-08T23:41:27.256539+0000","last_peered":"2024-05-08T23:41:27.256539+0000","last_clean":"2024-05-08T23:41:27.256539+0000","last_became_active":"2024-05-08T23:32:10.358558+0000","last_became_peered":"2024-05-08T23:32:10.358558+0000","last_unstale":"2024-05-08T23:41:27.256539+0000","last_undegraded":"2024-05-08T23:41:27.256539+0000","last_fullsized":"2024-05-08T23:41:27.256539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6396","last_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_deep_scrub":"23'6396","last_deep_scrub_stamp":"2024-05-08T23:41:27.256513+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.256513+0000","objects_scrubbed":0,"log_size":6396,"log_dups_size":0,"ondisk_log_size":6396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:10.352345+0000","scrub_duration":0.028481303999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96421,"num_write_kb":749546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'6","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.854006+0000","last_change":"2024-05-08T23:41:24.854120+0000","last_active":"2024-05-08T23:41:24.854006+0000","last_peered":"2024-05-08T23:41:24.854006+0000","last_clean":"2024-05-08T23:41:24.854006+0000","last_became_active":"2024-05-08T23:32:09.888144+0000","last_became_peered":"2024-05-08T23:32:09.888144+0000","last_unstale":"2024-05-08T23:41:24.854006+0000","last_undegraded":"2024-05-08T23:41:24.854006+0000","last_fullsized":"2024-05-08T23:41:24.854006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-05-08T23:41:24.853976+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.853976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:57:46.648846+0000","scrub_duration":0.0020858780000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.762775+0000","last_change":"2024-05-08T23:41:28.762886+0000","last_active":"2024-05-08T23:41:28.762775+0000","last_peered":"2024-05-08T23:41:28.762775+0000","last_clean":"2024-05-08T23:41:28.762775+0000","last_became_active":"2024-05-08T23:32:09.895609+0000","last_became_peered":"2024-05-08T23:32:09.895609+0000","last_unstale":"2024-05-08T23:41:28.762775+0000","last_undegraded":"2024-05-08T23:41:28.762775+0000","last_fullsized":"2024-05-08T23:41:28.762775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:28.762740+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.762740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:30.867016+0000","scrub_duration":0.0025176629999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'7015","reported_seq":7043,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:35.290975+0000","last_change":"2024-05-08T23:41:35.291126+0000","last_active":"2024-05-08T23:41:35.290975+0000","last_peered":"2024-05-08T23:41:35.290975+0000","last_clean":"2024-05-08T23:41:35.290975+0000","last_became_active":"2024-05-08T23:32:10.343910+0000","last_became_peered":"2024-05-08T23:32:10.343910+0000","last_unstale":"2024-05-08T23:41:35.290975+0000","last_undegraded":"2024-05-08T23:41:35.290975+0000","last_fullsized":"2024-05-08T23:41:35.290975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7015","last_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_deep_scrub":"23'7015","last_deep_scrub_stamp":"2024-05-08T23:41:35.290948+0000","last_clean_scrub_stamp":"2024-05-08T23:41:35.290948+0000","objects_scrubbed":0,"log_size":7015,"log_dups_size":0,"ondisk_log_size":7015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:46:19.726891+0000","scrub_duration":0.038503554000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105831,"num_write_kb":825639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'87","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:18.199741+0000","last_change":"2024-05-08T23:41:18.199836+0000","last_active":"2024-05-08T23:41:18.199741+0000","last_peered":"2024-05-08T23:41:18.199741+0000","last_clean":"2024-05-08T23:41:18.199741+0000","last_became_active":"2024-05-08T23:32:09.888641+0000","last_became_peered":"2024-05-08T23:32:09.888641+0000","last_unstale":"2024-05-08T23:41:18.199741+0000","last_undegraded":"2024-05-08T23:41:18.199741+0000","last_fullsized":"2024-05-08T23:41:18.199741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'87","last_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_deep_scrub":"23'87","last_deep_scrub_stamp":"2024-05-08T23:41:18.199711+0000","last_clean_scrub_stamp":"2024-05-08T23:41:18.199711+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:53.571638+0000","scrub_duration":0.0083734289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'6431","reported_seq":6459,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:37.929079+0000","last_change":"2024-05-08T23:41:37.929216+0000","last_active":"2024-05-08T23:41:37.929079+0000","last_peered":"2024-05-08T23:41:37.929079+0000","last_clean":"2024-05-08T23:41:37.929079+0000","last_became_active":"2024-05-08T23:32:10.362029+0000","last_became_peered":"2024-05-08T23:32:10.362029+0000","last_unstale":"2024-05-08T23:41:37.929079+0000","last_undegraded":"2024-05-08T23:41:37.929079+0000","last_fullsized":"2024-05-08T23:41:37.929079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6431","last_scrub_stamp":"2024-05-08T23:41:37.929053+0000","last_deep_scrub":"23'6431","last_deep_scrub_stamp":"2024-05-08T23:41:37.929053+0000","last_clean_scrub_stamp":"2024-05-08T23:41:37.929053+0000","objects_scrubbed":0,"log_size":6431,"log_dups_size":0,"ondisk_log_size":6431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:54:30.984973+0000","scrub_duration":0.027051103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96750,"num_write_kb":747115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'9","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:21.238040+0000","last_change":"2024-05-08T23:41:21.238150+0000","last_active":"2024-05-08T23:41:21.238040+0000","last_peered":"2024-05-08T23:41:21.238040+0000","last_clean":"2024-05-08T23:41:21.238040+0000","last_became_active":"2024-05-08T23:32:09.887341+0000","last_became_peered":"2024-05-08T23:32:09.887341+0000","last_unstale":"2024-05-08T23:41:21.238040+0000","last_undegraded":"2024-05-08T23:41:21.238040+0000","last_fullsized":"2024-05-08T23:41:21.238040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9","last_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_deep_scrub":"23'9","last_deep_scrub_stamp":"2024-05-08T23:41:21.238014+0000","last_clean_scrub_stamp":"2024-05-08T23:41:21.238014+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:43.382177+0000","scrub_duration":0.013372182999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'5458","reported_seq":5486,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:50.323706+0000","last_change":"2024-05-08T23:41:50.323889+0000","last_active":"2024-05-08T23:41:50.323706+0000","last_peered":"2024-05-08T23:41:50.323706+0000","last_clean":"2024-05-08T23:41:50.323706+0000","last_became_active":"2024-05-08T23:32:10.355758+0000","last_became_peered":"2024-05-08T23:32:10.355758+0000","last_unstale":"2024-05-08T23:41:50.323706+0000","last_undegraded":"2024-05-08T23:41:50.323706+0000","last_fullsized":"2024-05-08T23:41:50.323706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5458","last_scrub_stamp":"2024-05-08T23:41:50.323652+0000","last_deep_scrub":"23'5458","last_deep_scrub_stamp":"2024-05-08T23:41:50.323652+0000","last_clean_scrub_stamp":"2024-05-08T23:41:50.323652+0000","objects_scrubbed":0,"log_size":5458,"log_dups_size":0,"ondisk_log_size":5458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:02.615860+0000","scrub_duration":0.032782893,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82015,"num_write_kb":631614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.864949+0000","last_change":"2024-05-08T23:41:36.865073+0000","last_active":"2024-05-08T23:41:36.864949+0000","last_peered":"2024-05-08T23:41:36.864949+0000","last_clean":"2024-05-08T23:41:36.864949+0000","last_became_active":"2024-05-08T23:32:09.896930+0000","last_became_peered":"2024-05-08T23:32:09.896930+0000","last_unstale":"2024-05-08T23:41:36.864949+0000","last_undegraded":"2024-05-08T23:41:36.864949+0000","last_fullsized":"2024-05-08T23:41:36.864949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:36.864920+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.864920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:52.653980+0000","scrub_duration":0.0019745399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'5844","reported_seq":5872,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.531828+0000","last_change":"2024-05-08T23:41:27.531961+0000","last_active":"2024-05-08T23:41:27.531828+0000","last_peered":"2024-05-08T23:41:27.531828+0000","last_clean":"2024-05-08T23:41:27.531828+0000","last_became_active":"2024-05-08T23:32:10.346573+0000","last_became_peered":"2024-05-08T23:32:10.346573+0000","last_unstale":"2024-05-08T23:41:27.531828+0000","last_undegraded":"2024-05-08T23:41:27.531828+0000","last_fullsized":"2024-05-08T23:41:27.531828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5844","last_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_deep_scrub":"23'5844","last_deep_scrub_stamp":"2024-05-08T23:41:27.531800+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.531800+0000","objects_scrubbed":0,"log_size":5844,"log_dups_size":0,"ondisk_log_size":5844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:04:04.924146+0000","scrub_duration":0.023550181999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88164,"num_write_kb":684166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.185501+0000","last_change":"2024-05-08T23:41:25.185636+0000","last_active":"2024-05-08T23:41:25.185501+0000","last_peered":"2024-05-08T23:41:25.185501+0000","last_clean":"2024-05-08T23:41:25.185501+0000","last_became_active":"2024-05-08T23:32:09.886430+0000","last_became_peered":"2024-05-08T23:32:09.886430+0000","last_unstale":"2024-05-08T23:41:25.185501+0000","last_undegraded":"2024-05-08T23:41:25.185501+0000","last_fullsized":"2024-05-08T23:41:25.185501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_deep_scrub":"23'2","last_deep_scrub_stamp":"2024-05-08T23:41:25.185475+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.185475+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:27.661632+0000","scrub_duration":0.009407884,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'6488","reported_seq":6516,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.822830+0000","last_change":"2024-05-08T23:41:40.822968+0000","last_active":"2024-05-08T23:41:40.822830+0000","last_peered":"2024-05-08T23:41:40.822830+0000","last_clean":"2024-05-08T23:41:40.822830+0000","last_became_active":"2024-05-08T23:32:10.347435+0000","last_became_peered":"2024-05-08T23:32:10.347435+0000","last_unstale":"2024-05-08T23:41:40.822830+0000","last_undegraded":"2024-05-08T23:41:40.822830+0000","last_fullsized":"2024-05-08T23:41:40.822830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6488","last_scrub_stamp":"2024-05-08T23:41:40.822802+0000","last_deep_scrub":"23'6488","last_deep_scrub_stamp":"2024-05-08T23:41:40.822802+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.822802+0000","objects_scrubbed":0,"log_size":6488,"log_dups_size":0,"ondisk_log_size":6488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:38:49.757048+0000","scrub_duration":0.030908683999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98030,"num_write_kb":762965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:31.230555+0000","last_change":"2024-05-08T23:41:31.230705+0000","last_active":"2024-05-08T23:41:31.230555+0000","last_peered":"2024-05-08T23:41:31.230555+0000","last_clean":"2024-05-08T23:41:31.230555+0000","last_became_active":"2024-05-08T23:32:09.891157+0000","last_became_peered":"2024-05-08T23:32:09.891157+0000","last_unstale":"2024-05-08T23:41:31.230555+0000","last_undegraded":"2024-05-08T23:41:31.230555+0000","last_fullsized":"2024-05-08T23:41:31.230555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:31.230527+0000","last_clean_scrub_stamp":"2024-05-08T23:41:31.230527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:50.791503+0000","scrub_duration":0.0091231479999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'5341","reported_seq":5369,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:30.944446+0000","last_change":"2024-05-08T23:41:30.944603+0000","last_active":"2024-05-08T23:41:30.944446+0000","last_peered":"2024-05-08T23:41:30.944446+0000","last_clean":"2024-05-08T23:41:30.944446+0000","last_became_active":"2024-05-08T23:32:10.349057+0000","last_became_peered":"2024-05-08T23:32:10.349057+0000","last_unstale":"2024-05-08T23:41:30.944446+0000","last_undegraded":"2024-05-08T23:41:30.944446+0000","last_fullsized":"2024-05-08T23:41:30.944446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5341","last_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_deep_scrub":"23'5341","last_deep_scrub_stamp":"2024-05-08T23:41:30.944419+0000","last_clean_scrub_stamp":"2024-05-08T23:41:30.944419+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:32:37.870106+0000","scrub_duration":0.025456260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80213,"num_write_kb":621222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'687","reported_seq":716,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:25.835319+0000","last_change":"2024-05-08T23:41:25.835429+0000","last_active":"2024-05-08T23:41:25.835319+0000","last_peered":"2024-05-08T23:41:25.835319+0000","last_clean":"2024-05-08T23:41:25.835319+0000","last_became_active":"2024-05-08T23:32:09.891123+0000","last_became_peered":"2024-05-08T23:32:09.891123+0000","last_unstale":"2024-05-08T23:41:25.835319+0000","last_undegraded":"2024-05-08T23:41:25.835319+0000","last_fullsized":"2024-05-08T23:41:25.835319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'687","last_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_deep_scrub":"23'687","last_deep_scrub_stamp":"2024-05-08T23:41:25.835291+0000","last_clean_scrub_stamp":"2024-05-08T23:41:25.835291+0000","objects_scrubbed":2,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:22:44.548476+0000","scrub_duration":0.021024537999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":681,"num_write_kb":4628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'5938","reported_seq":5966,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:53.242233+0000","last_change":"2024-05-08T23:41:53.242369+0000","last_active":"2024-05-08T23:41:53.242233+0000","last_peered":"2024-05-08T23:41:53.242233+0000","last_clean":"2024-05-08T23:41:53.242233+0000","last_became_active":"2024-05-08T23:32:10.357798+0000","last_became_peered":"2024-05-08T23:32:10.357798+0000","last_unstale":"2024-05-08T23:41:53.242233+0000","last_undegraded":"2024-05-08T23:41:53.242233+0000","last_fullsized":"2024-05-08T23:41:53.242233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5938","last_scrub_stamp":"2024-05-08T23:41:53.242203+0000","last_deep_scrub":"23'5938","last_deep_scrub_stamp":"2024-05-08T23:41:53.242203+0000","last_clean_scrub_stamp":"2024-05-08T23:41:53.242203+0000","objects_scrubbed":0,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:10:29.838167+0000","scrub_duration":0.032392742000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89362,"num_write_kb":689576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:24.520446+0000","last_change":"2024-05-08T23:41:24.520818+0000","last_active":"2024-05-08T23:41:24.520446+0000","last_peered":"2024-05-08T23:41:24.520446+0000","last_clean":"2024-05-08T23:41:24.520446+0000","last_became_active":"2024-05-08T23:32:09.885212+0000","last_became_peered":"2024-05-08T23:32:09.885212+0000","last_unstale":"2024-05-08T23:41:24.520446+0000","last_undegraded":"2024-05-08T23:41:24.520446+0000","last_fullsized":"2024-05-08T23:41:24.520446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-05-08T23:41:24.520417+0000","last_clean_scrub_stamp":"2024-05-08T23:41:24.520417+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:56:30.649499+0000","scrub_duration":0.0021012050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'6544","reported_seq":6572,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.866623+0000","last_change":"2024-05-08T23:41:19.866792+0000","last_active":"2024-05-08T23:41:19.866623+0000","last_peered":"2024-05-08T23:41:19.866623+0000","last_clean":"2024-05-08T23:41:19.866623+0000","last_became_active":"2024-05-08T23:32:10.353849+0000","last_became_peered":"2024-05-08T23:32:10.353849+0000","last_unstale":"2024-05-08T23:41:19.866623+0000","last_undegraded":"2024-05-08T23:41:19.866623+0000","last_fullsized":"2024-05-08T23:41:19.866623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6544","last_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_deep_scrub":"23'6544","last_deep_scrub_stamp":"2024-05-08T23:41:19.866594+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.866594+0000","objects_scrubbed":0,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:33.883813+0000","scrub_duration":0.03064476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98339,"num_write_kb":766473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:28.262007+0000","last_change":"2024-05-08T23:41:28.262123+0000","last_active":"2024-05-08T23:41:28.262007+0000","last_peered":"2024-05-08T23:41:28.262007+0000","last_clean":"2024-05-08T23:41:28.262007+0000","last_became_active":"2024-05-08T23:32:09.890225+0000","last_became_peered":"2024-05-08T23:32:09.890225+0000","last_unstale":"2024-05-08T23:41:28.262007+0000","last_undegraded":"2024-05-08T23:41:28.262007+0000","last_fullsized":"2024-05-08T23:41:28.262007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-05-08T23:41:28.261980+0000","last_clean_scrub_stamp":"2024-05-08T23:41:28.261980+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:32:56.437680+0000","scrub_duration":0.018641068,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'6824","reported_seq":6852,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:36.871008+0000","last_change":"2024-05-08T23:41:36.871129+0000","last_active":"2024-05-08T23:41:36.871008+0000","last_peered":"2024-05-08T23:41:36.871008+0000","last_clean":"2024-05-08T23:41:36.871008+0000","last_became_active":"2024-05-08T23:32:10.350884+0000","last_became_peered":"2024-05-08T23:32:10.350884+0000","last_unstale":"2024-05-08T23:41:36.871008+0000","last_undegraded":"2024-05-08T23:41:36.871008+0000","last_fullsized":"2024-05-08T23:41:36.871008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6824","last_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_deep_scrub":"23'6824","last_deep_scrub_stamp":"2024-05-08T23:41:36.870982+0000","last_clean_scrub_stamp":"2024-05-08T23:41:36.870982+0000","objects_scrubbed":0,"log_size":6824,"log_dups_size":0,"ondisk_log_size":6824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:01:55.226753+0000","scrub_duration":0.026360792000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102945,"num_write_kb":800662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:19.492931+0000","last_change":"2024-05-08T23:41:19.492931+0000","last_active":"2024-05-08T23:41:19.492931+0000","last_peered":"2024-05-08T23:41:19.492931+0000","last_clean":"2024-05-08T23:41:19.492931+0000","last_became_active":"2024-05-08T23:32:09.880957+0000","last_became_peered":"2024-05-08T23:32:09.880957+0000","last_unstale":"2024-05-08T23:41:19.492931+0000","last_undegraded":"2024-05-08T23:41:19.492931+0000","last_fullsized":"2024-05-08T23:41:19.492931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-05-08T23:41:19.492750+0000","last_clean_scrub_stamp":"2024-05-08T23:41:19.492750+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:19:25.643838+0000","scrub_duration":0.0040015720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'7700","reported_seq":7728,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:44.834733+0000","last_change":"2024-05-08T23:41:44.834881+0000","last_active":"2024-05-08T23:41:44.834733+0000","last_peered":"2024-05-08T23:41:44.834733+0000","last_clean":"2024-05-08T23:41:44.834733+0000","last_became_active":"2024-05-08T23:32:10.354247+0000","last_became_peered":"2024-05-08T23:32:10.354247+0000","last_unstale":"2024-05-08T23:41:44.834733+0000","last_undegraded":"2024-05-08T23:41:44.834733+0000","last_fullsized":"2024-05-08T23:41:44.834733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7700","last_scrub_stamp":"2024-05-08T23:41:44.834706+0000","last_deep_scrub":"23'7700","last_deep_scrub_stamp":"2024-05-08T23:41:44.834706+0000","last_clean_scrub_stamp":"2024-05-08T23:41:44.834706+0000","objects_scrubbed":0,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:04.647359+0000","scrub_duration":0.028714782000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115971,"num_write_kb":898374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"23'7806","reported_seq":7834,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:48.199041+0000","last_change":"2024-05-08T23:41:48.199175+0000","last_active":"2024-05-08T23:41:48.199041+0000","last_peered":"2024-05-08T23:41:48.199041+0000","last_clean":"2024-05-08T23:41:48.199041+0000","last_became_active":"2024-05-08T23:32:10.351586+0000","last_became_peered":"2024-05-08T23:32:10.351586+0000","last_unstale":"2024-05-08T23:41:48.199041+0000","last_undegraded":"2024-05-08T23:41:48.199041+0000","last_fullsized":"2024-05-08T23:41:48.199041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7806","last_scrub_stamp":"2024-05-08T23:41:48.199014+0000","last_deep_scrub":"23'7806","last_deep_scrub_stamp":"2024-05-08T23:41:48.199014+0000","last_clean_scrub_stamp":"2024-05-08T23:41:48.199014+0000","objects_scrubbed":0,"log_size":7806,"log_dups_size":0,"ondisk_log_size":7806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:08:09.389806+0000","scrub_duration":0.033413879,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117565,"num_write_kb":911398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"23'6574","reported_seq":6602,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:27.586969+0000","last_change":"2024-05-08T23:41:27.587096+0000","last_active":"2024-05-08T23:41:27.586969+0000","last_peered":"2024-05-08T23:41:27.586969+0000","last_clean":"2024-05-08T23:41:27.586969+0000","last_became_active":"2024-05-08T23:32:10.345303+0000","last_became_peered":"2024-05-08T23:32:10.345303+0000","last_unstale":"2024-05-08T23:41:27.586969+0000","last_undegraded":"2024-05-08T23:41:27.586969+0000","last_fullsized":"2024-05-08T23:41:27.586969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6574","last_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_deep_scrub":"23'6574","last_deep_scrub_stamp":"2024-05-08T23:41:27.586941+0000","last_clean_scrub_stamp":"2024-05-08T23:41:27.586941+0000","objects_scrubbed":0,"log_size":6574,"log_dups_size":0,"ondisk_log_size":6574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:35.348883+0000","scrub_duration":0.027947453000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99039,"num_write_kb":765881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"23'8085","reported_seq":8113,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:45.002519+0000","last_change":"2024-05-08T23:41:45.002673+0000","last_active":"2024-05-08T23:41:45.002519+0000","last_peered":"2024-05-08T23:41:45.002519+0000","last_clean":"2024-05-08T23:41:45.002519+0000","last_became_active":"2024-05-08T23:32:10.351984+0000","last_became_peered":"2024-05-08T23:32:10.351984+0000","last_unstale":"2024-05-08T23:41:45.002519+0000","last_undegraded":"2024-05-08T23:41:45.002519+0000","last_fullsized":"2024-05-08T23:41:45.002519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8085","last_scrub_stamp":"2024-05-08T23:41:45.002491+0000","last_deep_scrub":"23'8085","last_deep_scrub_stamp":"2024-05-08T23:41:45.002491+0000","last_clean_scrub_stamp":"2024-05-08T23:41:45.002491+0000","objects_scrubbed":0,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:18.725445+0000","scrub_duration":0.037877398,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121835,"num_write_kb":935071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"23'5657","reported_seq":5685,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:51.330745+0000","last_change":"2024-05-08T23:41:51.330886+0000","last_active":"2024-05-08T23:41:51.330745+0000","last_peered":"2024-05-08T23:41:51.330745+0000","last_clean":"2024-05-08T23:41:51.330745+0000","last_became_active":"2024-05-08T23:32:10.350062+0000","last_became_peered":"2024-05-08T23:32:10.350062+0000","last_unstale":"2024-05-08T23:41:51.330745+0000","last_undegraded":"2024-05-08T23:41:51.330745+0000","last_fullsized":"2024-05-08T23:41:51.330745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5657","last_scrub_stamp":"2024-05-08T23:41:51.330715+0000","last_deep_scrub":"23'5657","last_deep_scrub_stamp":"2024-05-08T23:41:51.330715+0000","last_clean_scrub_stamp":"2024-05-08T23:41:51.330715+0000","objects_scrubbed":0,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:52.258234+0000","scrub_duration":0.027254093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85225,"num_write_kb":663319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"23'6017","reported_seq":6045,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:42.876533+0000","last_change":"2024-05-08T23:41:42.876674+0000","last_active":"2024-05-08T23:41:42.876533+0000","last_peered":"2024-05-08T23:41:42.876533+0000","last_clean":"2024-05-08T23:41:42.876533+0000","last_became_active":"2024-05-08T23:32:10.354504+0000","last_became_peered":"2024-05-08T23:32:10.354504+0000","last_unstale":"2024-05-08T23:41:42.876533+0000","last_undegraded":"2024-05-08T23:41:42.876533+0000","last_fullsized":"2024-05-08T23:41:42.876533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6017","last_scrub_stamp":"2024-05-08T23:41:42.876506+0000","last_deep_scrub":"23'6017","last_deep_scrub_stamp":"2024-05-08T23:41:42.876506+0000","last_clean_scrub_stamp":"2024-05-08T23:41:42.876506+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:47.973525+0000","scrub_duration":0.028109640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90906,"num_write_kb":707987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"23'7053","reported_seq":7081,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:41.364068+0000","last_change":"2024-05-08T23:41:41.364215+0000","last_active":"2024-05-08T23:41:41.364068+0000","last_peered":"2024-05-08T23:41:41.364068+0000","last_clean":"2024-05-08T23:41:41.364068+0000","last_became_active":"2024-05-08T23:32:10.348637+0000","last_became_peered":"2024-05-08T23:32:10.348637+0000","last_unstale":"2024-05-08T23:41:41.364068+0000","last_undegraded":"2024-05-08T23:41:41.364068+0000","last_fullsized":"2024-05-08T23:41:41.364068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7053","last_scrub_stamp":"2024-05-08T23:41:41.364041+0000","last_deep_scrub":"23'7053","last_deep_scrub_stamp":"2024-05-08T23:41:41.364041+0000","last_clean_scrub_stamp":"2024-05-08T23:41:41.364041+0000","objects_scrubbed":0,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:14.720054+0000","scrub_duration":0.026686147,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106470,"num_write_kb":822064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"23'5666","reported_seq":5694,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:42:00.288197+0000","last_change":"2024-05-08T23:42:00.288355+0000","last_active":"2024-05-08T23:42:00.288197+0000","last_peered":"2024-05-08T23:42:00.288197+0000","last_clean":"2024-05-08T23:42:00.288197+0000","last_became_active":"2024-05-08T23:32:10.354704+0000","last_became_peered":"2024-05-08T23:32:10.354704+0000","last_unstale":"2024-05-08T23:42:00.288197+0000","last_undegraded":"2024-05-08T23:42:00.288197+0000","last_fullsized":"2024-05-08T23:42:00.288197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5666","last_scrub_stamp":"2024-05-08T23:42:00.288170+0000","last_deep_scrub":"23'5666","last_deep_scrub_stamp":"2024-05-08T23:42:00.288170+0000","last_clean_scrub_stamp":"2024-05-08T23:42:00.288170+0000","objects_scrubbed":0,"log_size":5666,"log_dups_size":0,"ondisk_log_size":5666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:30.936606+0000","scrub_duration":0.028024278999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84987,"num_write_kb":658683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"23'7698","reported_seq":7726,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:39.838491+0000","last_change":"2024-05-08T23:41:39.838619+0000","last_active":"2024-05-08T23:41:39.838491+0000","last_peered":"2024-05-08T23:41:39.838491+0000","last_clean":"2024-05-08T23:41:39.838491+0000","last_became_active":"2024-05-08T23:32:10.353871+0000","last_became_peered":"2024-05-08T23:32:10.353871+0000","last_unstale":"2024-05-08T23:41:39.838491+0000","last_undegraded":"2024-05-08T23:41:39.838491+0000","last_fullsized":"2024-05-08T23:41:39.838491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7698","last_scrub_stamp":"2024-05-08T23:41:39.838465+0000","last_deep_scrub":"23'7698","last_deep_scrub_stamp":"2024-05-08T23:41:39.838465+0000","last_clean_scrub_stamp":"2024-05-08T23:41:39.838465+0000","objects_scrubbed":0,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:44:07.399697+0000","scrub_duration":0.016337357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115988,"num_write_kb":897700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'5557","reported_seq":5585,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:40.871363+0000","last_change":"2024-05-08T23:41:40.871507+0000","last_active":"2024-05-08T23:41:40.871363+0000","last_peered":"2024-05-08T23:41:40.871363+0000","last_clean":"2024-05-08T23:41:40.871363+0000","last_became_active":"2024-05-08T23:32:10.352746+0000","last_became_peered":"2024-05-08T23:32:10.352746+0000","last_unstale":"2024-05-08T23:41:40.871363+0000","last_undegraded":"2024-05-08T23:41:40.871363+0000","last_fullsized":"2024-05-08T23:41:40.871363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5557","last_scrub_stamp":"2024-05-08T23:41:40.871335+0000","last_deep_scrub":"23'5557","last_deep_scrub_stamp":"2024-05-08T23:41:40.871335+0000","last_clean_scrub_stamp":"2024-05-08T23:41:40.871335+0000","objects_scrubbed":0,"log_size":5557,"log_dups_size":0,"ondisk_log_size":5557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:39:03.187327+0000","scrub_duration":0.026704393999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83541,"num_write_kb":649443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'6393","reported_seq":6421,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:32.942026+0000","last_change":"2024-05-08T23:41:32.942160+0000","last_active":"2024-05-08T23:41:32.942026+0000","last_peered":"2024-05-08T23:41:32.942026+0000","last_clean":"2024-05-08T23:41:32.942026+0000","last_became_active":"2024-05-08T23:32:10.356867+0000","last_became_peered":"2024-05-08T23:32:10.356867+0000","last_unstale":"2024-05-08T23:41:32.942026+0000","last_undegraded":"2024-05-08T23:41:32.942026+0000","last_fullsized":"2024-05-08T23:41:32.942026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6393","last_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_deep_scrub":"23'6393","last_deep_scrub_stamp":"2024-05-08T23:41:32.942001+0000","last_clean_scrub_stamp":"2024-05-08T23:41:32.942001+0000","objects_scrubbed":0,"log_size":6393,"log_dups_size":0,"ondisk_log_size":6393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:28.602761+0000","scrub_duration":0.037578449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96164,"num_write_kb":744178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"23'7241","reported_seq":7269,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:57.109581+0000","last_change":"2024-05-08T23:41:57.109725+0000","last_active":"2024-05-08T23:41:57.109581+0000","last_peered":"2024-05-08T23:41:57.109581+0000","last_clean":"2024-05-08T23:41:57.109581+0000","last_became_active":"2024-05-08T23:32:10.347729+0000","last_became_peered":"2024-05-08T23:32:10.347729+0000","last_unstale":"2024-05-08T23:41:57.109581+0000","last_undegraded":"2024-05-08T23:41:57.109581+0000","last_fullsized":"2024-05-08T23:41:57.109581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7241","last_scrub_stamp":"2024-05-08T23:41:57.109551+0000","last_deep_scrub":"23'7241","last_deep_scrub_stamp":"2024-05-08T23:41:57.109551+0000","last_clean_scrub_stamp":"2024-05-08T23:41:57.109551+0000","objects_scrubbed":0,"log_size":7241,"log_dups_size":0,"ondisk_log_size":7241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:25:14.284887+0000","scrub_duration":0.037822815000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109160,"num_write_kb":852033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"23'6810","reported_seq":6838,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:52.329669+0000","last_change":"2024-05-08T23:41:52.329804+0000","last_active":"2024-05-08T23:41:52.329669+0000","last_peered":"2024-05-08T23:41:52.329669+0000","last_clean":"2024-05-08T23:41:52.329669+0000","last_became_active":"2024-05-08T23:32:10.350243+0000","last_became_peered":"2024-05-08T23:32:10.350243+0000","last_unstale":"2024-05-08T23:41:52.329669+0000","last_undegraded":"2024-05-08T23:41:52.329669+0000","last_fullsized":"2024-05-08T23:41:52.329669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6810","last_scrub_stamp":"2024-05-08T23:41:52.329638+0000","last_deep_scrub":"23'6810","last_deep_scrub_stamp":"2024-05-08T23:41:52.329638+0000","last_clean_scrub_stamp":"2024-05-08T23:41:52.329638+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:07:22.093136+0000","scrub_duration":0.029955665999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102452,"num_write_kb":794167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"23'6394","reported_seq":6422,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T23:41:46.180217+0000","last_change":"2024-05-08T23:41:46.180368+0000","last_active":"2024-05-08T23:41:46.180217+0000","last_peered":"2024-05-08T23:41:46.180217+0000","last_clean":"2024-05-08T23:41:46.180217+0000","last_became_active":"2024-05-08T23:32:10.357421+0000","last_became_peered":"2024-05-08T23:32:10.357421+0000","last_unstale":"2024-05-08T23:41:46.180217+0000","last_undegraded":"2024-05-08T23:41:46.180217+0000","last_fullsized":"2024-05-08T23:41:46.180217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6394","last_scrub_stamp":"2024-05-08T23:41:46.180190+0000","last_deep_scrub":"23'6394","last_deep_scrub_stamp":"2024-05-08T23:41:46.180190+0000","last_clean_scrub_stamp":"2024-05-08T23:41:46.180190+0000","objects_scrubbed":0,"log_size":6394,"log_dups_size":0,"ondisk_log_size":6394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:37:39.068684+0000","scrub_duration":0.028385853999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96406,"num_write_kb":752603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6318109,"num_write_kb":48997707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":419416,"ondisk_log_size":419416,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":17293737,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":24,"num_read_kb":101,"num_write":2683,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34652160,"data_stored":34587474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2678,"ondisk_log_size":2678,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617404,"kb_used_data":13108,"kb_used_omap":1,"kb_used_meta":604286,"kb_avail":93754436,"statfs":{"total":96636764160,"available":96004542464,"internally_reserved":0,"allocated":13422592,"data_stored":13261890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":618789323},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":8.0020000000000007,"15min":3.2799999999999998},"min":{"1min":0.29299999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":7.9000000000000004,"5min":71.385000000000005,"15min":71.385000000000005},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.536,"5min":7.2050000000000001,"15min":3.1040000000000001},"min":{"1min":0.34699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":15.487,"5min":37.293999999999997,"15min":37.293999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":4.0140000000000002,"15min":1.996},"min":{"1min":0.40000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.5350000000000001,"5min":26.004000000000001,"15min":26.004000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":7.3179999999999996,"15min":3.161},"min":{"1min":0.36799999999999999,"5min":0.31,"15min":0.31},"max":{"1min":7.9279999999999999,"5min":61.826000000000001,"15min":61.826000000000001},"last":0.621}]},{"osd":2,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":3.827,"15min":1.944},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.9589999999999996,"5min":19.164999999999999,"15min":26.152000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.609,"5min":4.7839999999999998,"15min":2.1850000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":7.375,"5min":26.084,"15min":26.084},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":6.6429999999999998,"15min":2.9350000000000001},"min":{"1min":0.45300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.2999999999999998,"5min":52.869,"15min":52.869},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.478,"5min":7.1310000000000002,"15min":2.911},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.35499999999999998},"max":{"1min":5.9299999999999997,"5min":56.459000000000003,"15min":56.459000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":4.4610000000000003,"15min":2.089},"min":{"1min":0.35799999999999998,"5min":0.32000000000000001,"15min":0.317},"max":{"1min":5.8109999999999999,"5min":53.552999999999997,"15min":53.552999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.129,"5min":3.5609999999999999,"15min":1.7310000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":7.8550000000000004,"5min":26.981000000000002,"15min":26.981000000000002},"last":0.439}]},{"osd":5,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":4.2480000000000002,"15min":2.1600000000000001},"min":{"1min":0.38300000000000001,"5min":0.35699999999999998,"15min":0.33100000000000002},"max":{"1min":10.327,"5min":25.116,"15min":63.732999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":6.2969999999999997,"15min":3.028},"min":{"1min":0.46100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":8.5020000000000007,"5min":37.552999999999997,"15min":62.558},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed May 8 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":2.0070000000000001,"5min":6.8129999999999997,"15min":3.0409999999999999},"min":{"1min":0.432,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":13.154999999999999,"5min":53.604999999999997,"15min":53.604999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.7629999999999999,"5min":4.4379999999999997,"15min":2.056},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":12.143000000000001,"5min":38.801000000000002,"15min":38.801000000000002},"last":0.52300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404896,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":400126,"kb_avail":93966944,"statfs":{"total":96636764160,"available":96222150656,"internally_reserved":0,"allocated":4874240,"data_stored":4710076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":409729479},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":6.3659999999999997,"15min":2.8140000000000001},"min":{"1min":0.36199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":8.0109999999999992,"5min":59.654000000000003,"15min":59.654000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":6.9820000000000002,"15min":3.1440000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.25800000000000001},"max":{"1min":3.9660000000000002,"5min":135.19800000000001,"15min":135.19800000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":5.9649999999999999,"15min":2.8879999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4629999999999992,"5min":45.768999999999998,"15min":63.268000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":5.2009999999999996,"15min":2.6269999999999998},"min":{"1min":0.43099999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":4.0460000000000003,"5min":41.673999999999999,"15min":63.057000000000002},"last":0.61499999999999999}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":4.0869999999999997,"15min":2.1139999999999999},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.28599999999999998},"max":{"1min":4.0099999999999998,"5min":41.445,"15min":41.445},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":4.1420000000000003,"15min":2.085},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.0880000000000001,"5min":41.487000000000002,"15min":41.487000000000002},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":4.3869999999999996,"15min":2.3119999999999998},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.9060000000000006,"5min":24.216999999999999,"15min":24.216999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":6.5350000000000001,"15min":3.2010000000000001},"min":{"1min":0.42499999999999999,"5min":0.373,"15min":0.373},"max":{"1min":8.8650000000000002,"5min":135.14500000000001,"15min":135.14500000000001},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":2.9929999999999999,"15min":1.466},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.069,"5min":20.913,"15min":20.913},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":3.2480000000000002,"15min":1.831},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.32600000000000001},"max":{"1min":4.1230000000000002,"5min":22.361000000000001,"15min":35.975999999999999},"last":0.63}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":5.9729999999999999,"15min":2.7229999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.4199999999999999,"5min":45.718000000000004,"15min":45.718000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":1.4850000000000001,"5min":4.3899999999999997,"15min":2.081},"min":{"1min":0.373,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.3130000000000006,"5min":23.105,"15min":23.105},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":3.5179999999999998,"15min":2.1019999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.32300000000000001},"max":{"1min":16.905999999999999,"5min":24.16,"15min":33.463000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.74,"5min":5.5640000000000001,"15min":2.4199999999999999},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":18.433,"5min":45.805,"15min":45.805},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400316,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":400062,"kb_avail":93971524,"statfs":{"total":96636764160,"available":96226840576,"internally_reserved":0,"allocated":249856,"data_stored":88188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":409663942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":5.8040000000000003,"15min":2.5310000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":3.6070000000000002,"5min":38.329000000000001,"15min":38.329000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":3.8109999999999999,"15min":2.1469999999999998},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.25900000000000001},"max":{"1min":0.98799999999999999,"5min":39.283999999999999,"15min":48.844999999999999},"last":0.33400000000000002}]},{"osd":2,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":3.7309999999999999,"15min":1.891},"min":{"1min":0.34200000000000003,"5min":0.26200000000000001,"15min":0.26100000000000001},"max":{"1min":1.2230000000000001,"5min":38.661999999999999,"15min":38.661999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":5.2709999999999999,"15min":2.3410000000000002},"min":{"1min":0.44,"5min":0.312,"15min":0.28599999999999998},"max":{"1min":3.9390000000000001,"5min":38.784999999999997,"15min":38.784999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":6.6470000000000002,"15min":2.972},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.27700000000000002},"max":{"1min":1.1379999999999999,"5min":46.390999999999998,"15min":52.712000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":7.0099999999999998,"15min":3.4780000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32100000000000001},"max":{"1min":1.2929999999999999,"5min":53.265999999999998,"15min":61.832000000000001},"last":0.502}]},{"osd":4,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":5.3079999999999998,"15min":2.7050000000000001},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":6.8620000000000001,"5min":34.500999999999998,"15min":45.267000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":5.4020000000000001,"15min":2.6099999999999999},"min":{"1min":0.52200000000000002,"5min":0.40500000000000003,"15min":0.34000000000000002},"max":{"1min":3.7810000000000001,"5min":34.347999999999999,"15min":43.353999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":5.0700000000000003,"15min":2.6989999999999998},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.8200000000000003,"5min":45.326999999999998,"15min":45.326999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.98099999999999998,"5min":6.5780000000000003,"15min":3.2839999999999998},"min":{"1min":0.60399999999999998,"5min":0.378,"15min":0.378},"max":{"1min":5.8310000000000004,"5min":46.246000000000002,"15min":46.246000000000002},"last":0.69699999999999995}]},{"osd":6,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":5.6740000000000004,"15min":3.0920000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.9780000000000002,"5min":46.710000000000001,"15min":46.710000000000001},"last":0.749},{"interface":"front","average":{"1min":0.97299999999999998,"5min":5.5620000000000003,"15min":2.9009999999999998},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.442},"max":{"1min":6.3860000000000001,"5min":37.802,"15min":42.603999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Wed May 8 23:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":7.7400000000000002,"15min":3.4689999999999999},"min":{"1min":0.504,"5min":0.42699999999999999,"15min":0.38100000000000001},"max":{"1min":4.6260000000000003,"5min":72.088999999999999,"15min":72.088999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":7.6180000000000003,"15min":3.7349999999999999},"min":{"1min":0.52800000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.6630000000000003,"5min":72.150000000000006,"15min":72.150000000000006},"last":0.80900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581380,"kb_used_data":4924,"kb_used_omap":1,"kb_used_meta":576446,"kb_avail":93790460,"statfs":{"total":96636764160,"available":96041431040,"internally_reserved":0,"allocated":5042176,"data_stored":4873338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590281158},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":5.5460000000000003,"15min":2.899},"min":{"1min":0.216,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":4.5140000000000002,"5min":47.241999999999997,"15min":101.899},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":4.5510000000000002,"15min":2.5579999999999998},"min":{"1min":0.28599999999999998,"5min":0.253,"15min":0.253},"max":{"1min":3.2690000000000001,"5min":38.814999999999998,"15min":78.897000000000006},"last":0.38400000000000001}]},{"osd":2,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":7.4269999999999996,"15min":3.5910000000000002},"min":{"1min":0.314,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.8660000000000001,"5min":134.16300000000001,"15min":134.16300000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":6.5880000000000001,"15min":3.3519999999999999},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.29999999999999999},"max":{"1min":2.3519999999999999,"5min":68.197999999999993,"15min":78.817999999999998},"last":0.441}]},{"osd":3,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.423,"5min":6.2320000000000002,"15min":3.383},"min":{"1min":0.35999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":24.239000000000001,"5min":59.796999999999997,"15min":101.929},"last":0.60199999999999998},{"interface":"front","average":{"1min":2.399,"5min":3.3879999999999999,"15min":2.3410000000000002},"min":{"1min":0.32300000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":23.664999999999999,"5min":23.664999999999999,"15min":95.200999999999993},"last":0.33200000000000002}]},{"osd":4,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":6.0060000000000002,"15min":3.2549999999999999},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.35399999999999998},"max":{"1min":6.9850000000000003,"5min":47.194000000000003,"15min":101.821},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.228,"5min":3.7610000000000001,"15min":2.3889999999999998},"min":{"1min":0.46600000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.5709999999999997,"5min":19.163,"15min":44.545999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":3.3410000000000002,"5min":6.1820000000000004,"15min":3.0310000000000001},"min":{"1min":0.44700000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":43.124000000000002,"5min":43.124000000000002,"15min":78.780000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":4.2110000000000003,"5min":5.843,"15min":3.0070000000000001},"min":{"1min":0.41399999999999998,"5min":0.40699999999999997,"15min":0.40200000000000002},"max":{"1min":43.109000000000002,"5min":55.957000000000001,"15min":55.957000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":5.8769999999999998,"15min":3.351},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":11.282999999999999,"5min":45.744999999999997,"15min":101.869},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.284,"5min":5.0599999999999996,"15min":2.944},"min":{"1min":0.48199999999999998,"5min":0.317,"15min":0.317},"max":{"1min":6.8890000000000002,"5min":38.545000000000002,"15min":78.647999999999996},"last":0.91600000000000004}]},{"osd":7,"last update":"Wed May 8 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.4509999999999996,"15min":2.9009999999999998},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":6.9580000000000002,"5min":37.725000000000001,"15min":78.736999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.482,"5min":3.7200000000000002,"15min":2.3279999999999998},"min":{"1min":0.48999999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.532,"5min":19.065000000000001,"15min":44.713999999999999},"last":0.68999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574971,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":338060,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":337726,"kb_avail":94033780,"statfs":{"total":96636764160,"available":96290590720,"internally_reserved":0,"allocated":331776,"data_stored":176007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":345831878},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":5.266,"15min":2.2570000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.20999999999999999},"max":{"1min":9.6880000000000006,"5min":44.238999999999997,"15min":44.238999999999997},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":4.6859999999999999,"15min":2.149},"min":{"1min":0.375,"5min":0.31900000000000001,"15min":0.27700000000000002},"max":{"1min":9.7699999999999996,"5min":44.353000000000002,"15min":44.353000000000002},"last":0.436}]},{"osd":1,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":5.0739999999999998,"15min":2.5710000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.255},"max":{"1min":1.9159999999999999,"5min":79.540999999999997,"15min":79.540999999999997},"last":0.33400000000000002},{"interface":"front","average":{"1min":1.2330000000000001,"5min":5.4859999999999998,"15min":2.524},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.318},"max":{"1min":7.0469999999999997,"5min":79.584000000000003,"15min":79.584000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":7.5650000000000004,"15min":3.516},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.27600000000000002},"max":{"1min":22.989999999999998,"5min":72.956999999999994,"15min":72.956999999999994},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.622,"5min":4.0359999999999996,"15min":2.0750000000000002},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.29199999999999998},"max":{"1min":1.6659999999999999,"5min":50.192999999999998,"15min":50.192999999999998},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.1959999999999997,"15min":2.363},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":29.513999999999999,"5min":41.488999999999997,"15min":41.488999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.1339999999999999,"5min":3.8719999999999999,"15min":2.0150000000000001},"min":{"1min":0.499,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":7.0970000000000004,"5min":23.951000000000001,"15min":23.951000000000001},"last":0.55000000000000004}]},{"osd":5,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":5.6399999999999997,"15min":2.8500000000000001},"min":{"1min":0.49199999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":12.858000000000001,"5min":32.779000000000003,"15min":32.779000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.853,"5min":5.0960000000000001,"15min":2.371},"min":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":12.723000000000001,"5min":30.152000000000001,"15min":30.152000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":4.8440000000000003,"15min":2.278},"min":{"1min":0.629,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":10.128,"5min":41.396999999999998,"15min":41.396999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.647,"5min":5.8680000000000003,"15min":2.605},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":9.3510000000000009,"5min":42.280999999999999,"15min":42.280999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Wed May 8 23:41:16 2024","interfaces":[{"interface":"back","average":{"1min":1.768,"5min":4.9199999999999999,"15min":2.3759999999999999},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.5679999999999996,"5min":24.623999999999999,"15min":24.623999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":2.2879999999999998,"5min":6.5650000000000004,"15min":2.96},"min":{"1min":0.51100000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.9250000000000007,"5min":42.875999999999998,"15min":42.875999999999998},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602964,"kb_used_data":8524,"kb_used_omap":1,"kb_used_meta":594430,"kb_avail":93768876,"statfs":{"total":96636764160,"available":96019329024,"internally_reserved":0,"allocated":8728576,"data_stored":8565287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":608696776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":5.181,"15min":2.605},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.22600000000000001},"max":{"1min":6.7359999999999998,"5min":53.103999999999999,"15min":67.305999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":1.5149999999999999,"5min":6.6479999999999997,"15min":3.464},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":14.592000000000001,"5min":39.784999999999997,"15min":106.934},"last":0.46000000000000002}]},{"osd":1,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":6.1829999999999998,"15min":2.7589999999999999},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.26600000000000001},"max":{"1min":14.182,"5min":76.628,"15min":76.628},"last":0.44},{"interface":"front","average":{"1min":2.0030000000000001,"5min":5.4720000000000004,"15min":2.6099999999999999},"min":{"1min":0.39900000000000002,"5min":0.27600000000000002,"15min":0.26200000000000001},"max":{"1min":14.577,"5min":54.991999999999997,"15min":54.991999999999997},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":7.0739999999999998,"15min":3.3050000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":65.519000000000005,"5min":65.519000000000005,"15min":65.519000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":1.375,"5min":4.75,"15min":2.4169999999999998},"min":{"1min":0.44900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.057,"5min":44.951000000000001,"15min":44.951000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.0999999999999996,"5min":7.9180000000000001,"15min":3.5569999999999999},"min":{"1min":0.40799999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":65.575000000000003,"5min":65.575000000000003,"15min":65.575000000000003},"last":0.80700000000000005},{"interface":"front","average":{"1min":3.726,"5min":7.548,"15min":3.137},"min":{"1min":0.47299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":31.77,"5min":68.238,"15min":68.238},"last":0.76600000000000001}]},{"osd":5,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.2990000000000004,"5min":6.6740000000000004,"15min":2.9620000000000002},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.38100000000000001},"max":{"1min":31.806000000000001,"5min":53.381,"15min":53.381},"last":0.63700000000000001},{"interface":"front","average":{"1min":3.157,"5min":6.7670000000000003,"15min":3.294},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":26.997,"5min":32.381,"15min":36.213999999999999},"last":0.747}]},{"osd":6,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":5.1429999999999998,"5min":8.0150000000000006,"15min":3.391},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":65.606999999999999,"5min":65.606999999999999,"15min":65.606999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.925,"5min":5.1440000000000001,"15min":2.464},"min":{"1min":0.57099999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":10.94,"5min":44.825000000000003,"15min":44.825000000000003},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed May 8 23:41:19 2024","interfaces":[{"interface":"back","average":{"1min":4.9100000000000001,"5min":9.0250000000000004,"15min":3.8580000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.375},"max":{"1min":65.638999999999996,"5min":65.638999999999996,"15min":65.638999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.421,"5min":6.2400000000000002,"15min":2.9540000000000002},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":6.681,"5min":53.182000000000002,"15min":53.182000000000002},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549220,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":544446,"kb_avail":93822620,"statfs":{"total":96636764160,"available":96074362880,"internally_reserved":0,"allocated":4878336,"data_stored":4709530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":557513158},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3580000000000001,"5min":6.1509999999999998,"15min":2.8980000000000001},"min":{"1min":0.32400000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.3620000000000001,"5min":87.162999999999997,"15min":87.162999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":2.9729999999999999,"5min":6.7389999999999999,"15min":3.109},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":10.945,"5min":92.325000000000003,"15min":92.325000000000003},"last":0.63700000000000001}]},{"osd":1,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7050000000000001,"5min":5.7030000000000003,"15min":2.9729999999999999},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":24.324999999999999,"5min":35.807000000000002,"15min":40.901000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":5.6130000000000004,"5min":5.867,"15min":2.8839999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":47.481999999999999,"5min":47.481999999999999,"15min":47.481999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.9279999999999999,"5min":5.4900000000000002,"15min":2.871},"min":{"1min":0.35799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":13.356999999999999,"5min":43.743000000000002,"15min":43.743000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":2.7730000000000001,"5min":4.8339999999999996,"15min":2.5529999999999999},"min":{"1min":0.42199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":11.509,"5min":25.616,"15min":58.899000000000001},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":3.6280000000000001,"5min":5.835,"15min":2.859},"min":{"1min":0.40899999999999997,"5min":0.318,"15min":0.318},"max":{"1min":24.460999999999999,"5min":43.664000000000001,"15min":43.664000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":4.3490000000000002,"5min":7.7370000000000001,"15min":3.4350000000000001},"min":{"1min":0.40699999999999997,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":31.451000000000001,"5min":111.221,"15min":111.221},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":4.6660000000000004,"15min":2.266},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.7229999999999999,"5min":43.497999999999998,"15min":51.871000000000002},"last":0.317},{"interface":"front","average":{"1min":1.8460000000000001,"5min":4.4720000000000004,"15min":2.0409999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.7100000000000009,"5min":34.161000000000001,"15min":34.161000000000001},"last":0.437}]},{"osd":6,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2930000000000001,"5min":4.5890000000000004,"15min":2.0680000000000001},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":11.436999999999999,"5min":32.198999999999998,"15min":32.198999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":2.2069999999999999,"5min":4.548,"15min":2.3849999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":11.083,"5min":43.774999999999999,"15min":51.932000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Wed May 8 23:41:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5590000000000002,"5min":4.0289999999999999,"15min":1.9630000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":7.4790000000000001,"5min":28.266999999999999,"15min":28.266999999999999},"last":0.501},{"interface":"front","average":{"1min":2.6930000000000001,"5min":4.7949999999999999,"15min":2.4729999999999999},"min":{"1min":0.50800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":14.221,"5min":44.783000000000001,"15min":51.978000000000002},"last":0.59099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554552,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":554302,"kb_avail":93817288,"statfs":{"total":96636764160,"available":96068902912,"internally_reserved":0,"allocated":245760,"data_stored":88118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567605702},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9710000000000001,"5min":5.4900000000000002,"15min":2.7029999999999998},"min":{"1min":0.28599999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":7.4160000000000004,"5min":42.609000000000002,"15min":42.609000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":1.8300000000000001,"5min":5.2009999999999996,"15min":2.3119999999999998},"min":{"1min":0.497,"5min":0.42399999999999999,"15min":0.30199999999999999},"max":{"1min":7.7690000000000001,"5min":44.439,"15min":44.439},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":4.9509999999999996,"15min":2.9039999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32100000000000001},"max":{"1min":7.2759999999999998,"5min":31.355,"15min":124.85599999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.347,"5min":4.8479999999999999,"15min":2.4580000000000002},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.317},"max":{"1min":7.327,"5min":31.385000000000002,"15min":41.060000000000002},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.8580000000000001,"5min":6.5110000000000001,"15min":2.9790000000000001},"min":{"1min":0.45600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":7.8650000000000002,"5min":44.247,"15min":44.247},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.7170000000000001,"5min":4.8680000000000003,"15min":2.3039999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.35499999999999998},"max":{"1min":7.5350000000000001,"5min":39.975000000000001,"15min":39.975000000000001},"last":0.68700000000000006}]},{"osd":3,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":5.0279999999999996,"5min":7.7690000000000001,"15min":3.1699999999999999},"min":{"1min":0.41999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":39.265000000000001,"5min":79.974000000000004,"15min":79.974000000000004},"last":0.63},{"interface":"front","average":{"1min":4.1520000000000001,"5min":7.9029999999999996,"15min":4.0640000000000001},"min":{"1min":0.46100000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":19.193000000000001,"5min":43.343000000000004,"15min":49.991999999999997},"last":0.82499999999999996}]},{"osd":4,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":5.1840000000000002,"15min":2.181},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":8.8119999999999994,"5min":43.305,"15min":43.305},"last":0.5},{"interface":"front","average":{"1min":1.4199999999999999,"5min":3.669,"15min":1.8049999999999999},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.3780000000000001,"5min":42.551000000000002,"15min":42.551000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":1.9830000000000001,"5min":5.4550000000000001,"15min":2.855},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.32300000000000001},"max":{"1min":7.8280000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.747},{"interface":"front","average":{"1min":2.0369999999999999,"5min":4.7329999999999997,"15min":2.306},"min":{"1min":0.45000000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.2289999999999992,"5min":46.505000000000003,"15min":46.505000000000003},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:41:18 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":5.6580000000000004,"15min":2.532},"min":{"1min":0.44500000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":15.589,"5min":43.262,"15min":43.262},"last":0.72199999999999998},{"interface":"front","average":{"1min":2.2690000000000001,"5min":6.3700000000000001,"15min":3.3159999999999998},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.40600000000000003},"max":{"1min":10.456,"5min":44.018000000000001,"15min":80.302000000000007},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":87979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8486912,"data_stored":8477259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4636672,"data_stored":4621502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4632576,"data_stored":4622048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:42:21.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T23:42:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.391+0000 7f52603b8700 1 Processor -- start 2024-05-08T23:42:21.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.391+0000 7f52603b8700 1 -- start start 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5258154970 0x7f525806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f525806b280 0x7f52580cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 0x7f52580cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525815b350 con 0x7f5258154970 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525815e8a0 con 0x7f525806b280 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.392+0000 7f52603b8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525815a5d0 con 0x7f52580caad0 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5258154970 0x7f525806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5258154970 0x7f525806ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38500/0 (socket says 172.21.15.33:38500) 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 -- 172.21.15.33:0/3058806182 learned_addr learned my addr 172.21.15.33:0/3058806182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525d953700 1 --2- 172.21.15.33:0/3058806182 >> v2:172.21.15.42:3300/0 conn(0x7f525806b280 0x7f52580cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e955700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 0x7f52580cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 -- 172.21.15.33:0/3058806182 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 msgr2=0x7f52580cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 0x7f52580cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 -- 172.21.15.33:0/3058806182 >> v2:172.21.15.42:3300/0 conn(0x7f525806b280 msgr2=0x7f52580cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 --2- 172.21.15.33:0/3058806182 >> v2:172.21.15.42:3300/0 conn(0x7f525806b280 0x7f52580cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 -- 172.21.15.33:0/3058806182 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525815e5c0 con 0x7f5258154970 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e955700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 0x7f52580cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525e154700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5258154970 0x7f525806ad40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52580cd6b0 tx=0x7f5254009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399d32519830fe74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525d152700 1 -- 172.21.15.33:0/3058806182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254011830 con 0x7f5258154970 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525d152700 1 -- 172.21.15.33:0/3058806182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254011990 con 0x7f5258154970 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.393+0000 7f525d152700 1 -- 172.21.15.33:0/3058806182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254011d20 con 0x7f5258154970 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 -- 172.21.15.33:0/3058806182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5258154970 msgr2=0x7f525806ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5258154970 0x7f525806ad40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52580cd6b0 tx=0x7f5254009bd0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 -- 172.21.15.33:0/3058806182 shutdown_connections 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5258154970 0x7f525806ad40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 --2- 172.21.15.33:0/3058806182 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f52580caad0 0x7f52580cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 --2- 172.21.15.33:0/3058806182 >> v2:172.21.15.42:3300/0 conn(0x7f525806b280 0x7f52580cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 -- 172.21.15.33:0/3058806182 >> 172.21.15.33:0/3058806182 conn(0x7f525806cd80 msgr2=0x7f525806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 -- 172.21.15.33:0/3058806182 shutdown_connections 2024-05-08T23:42:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.394+0000 7f52603b8700 1 -- 172.21.15.33:0/3058806182 wait complete. 2024-05-08T23:42:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.395+0000 7f52603b8700 1 Processor -- start 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.395+0000 7f52603b8700 1 -- start start 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f525806b280 0x7f52581e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5258154970 0x7f52581f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581e48d0 con 0x7f52580caad0 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581e3780 con 0x7f5258154970 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f52603b8700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52581e6520 con 0x7f525806b280 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f525d953700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f525d953700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38502/0 (socket says 172.21.15.33:38502) 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.396+0000 7f525d953700 1 -- 172.21.15.33:0/2466680284 learned_addr learned my addr 172.21.15.33:0/2466680284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525e955700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5258154970 0x7f52581f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525e154700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f525806b280 0x7f52581e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 -- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f525806b280 msgr2=0x7f52581e59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f525806b280 0x7f52581e59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 -- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5258154970 msgr2=0x7f52581f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5258154970 0x7f52581f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 -- 172.21.15.33:0/2466680284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5254009550 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f525d953700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f524800e900 tx=0x7f524800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248009690 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800c850 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800cc00 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5258159150 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.397+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52581e3c70 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.398+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525815ab10 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.399+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f5248013470 con 0x7f52580caad0 2024-05-08T23:42:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.399+0000 7f524effd700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 0x7f523c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.399+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f52480897d0 con 0x7f52580caad0 2024-05-08T23:42:21.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.399+0000 7f525e154700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 0x7f523c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.400+0000 7f525e154700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 0x7f523c06d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5254014040 tx=0x7f5254008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.404+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f52480548e0 con 0x7f52580caad0 2024-05-08T23:42:21.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.620+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f525806be90 con 0x7f52580caad0 2024-05-08T23:42:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.623+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f52480548e0 con 0x7f52580caad0 2024-05-08T23:42:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.623+0000 7f524effd700 1 -- 172.21.15.33:0/2466680284 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f52480170c0 con 0x7f52580caad0 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 msgr2=0x7f523c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 0x7f523c06d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5254014040 tx=0x7f5254008030 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 msgr2=0x7f52581f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f524800e900 tx=0x7f524800ecc0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 shutdown_connections 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f523c06ac90 0x7f523c06d150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5258154970 0x7f52581f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f525806b280 0x7f52581e59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 --2- 172.21.15.33:0/2466680284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52580caad0 0x7f52581f3d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.626+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 >> 172.21.15.33:0/2466680284 conn(0x7f525806cd80 msgr2=0x7f52581530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:21.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.627+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 shutdown_connections 2024-05-08T23:42:21.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.627+0000 7f52603b8700 1 -- 172.21.15.33:0/2466680284 wait complete. 2024-05-08T23:42:21.637 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:42:21.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:42:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.778+0000 7f2589cda700 1 Processor -- start 2024-05-08T23:42:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.778+0000 7f2589cda700 1 -- start start 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25841572c0 0x7f25840cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 0x7f25840c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841576a0 0x7f2584150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840cc430 con 0x7f25841572c0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840c3580 con 0x7f25840cc5d0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2589cda700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840c2fa0 con 0x7f25841576a0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25841572c0 0x7f25840cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f25841572c0 0x7f25840cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38514/0 (socket says 172.21.15.33:38514) 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 -- 172.21.15.33:0/3154499532 learned_addr learned my addr 172.21.15.33:0/3154499532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2583fff700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841576a0 0x7f2584150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2582ffd700 1 --2- 172.21.15.33:0/3154499532 >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 0x7f25840c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 -- 172.21.15.33:0/3154499532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841576a0 msgr2=0x7f2584150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841576a0 0x7f2584150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 -- 172.21.15.33:0/3154499532 >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 msgr2=0x7f25840c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 --2- 172.21.15.33:0/3154499532 >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 0x7f25840c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 -- 172.21.15.33:0/3154499532 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840ccaa0 con 0x7f25841572c0 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f2582ffd700 1 --2- 172.21.15.33:0/3154499532 >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 0x7f25840c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25837fe700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841572c0 0x7f25840cbef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25840c7490 tx=0x7f2578009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e15bb1e9109b6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25827fc700 1 -- 172.21.15.33:0/3154499532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800f8a0 con 0x7f25841572c0 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.779+0000 7f25827fc700 1 -- 172.21.15.33:0/3154499532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f257800fa00 con 0x7f25841572c0 2024-05-08T23:42:21.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f25827fc700 1 -- 172.21.15.33:0/3154499532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800fdb0 con 0x7f25841572c0 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 -- 172.21.15.33:0/3154499532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841572c0 msgr2=0x7f25840cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841572c0 0x7f25840cbef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25840c7490 tx=0x7f2578009bd0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 -- 172.21.15.33:0/3154499532 shutdown_connections 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841572c0 0x7f25840cbef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 --2- 172.21.15.33:0/3154499532 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841576a0 0x7f2584150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 --2- 172.21.15.33:0/3154499532 >> v2:172.21.15.42:3300/0 conn(0x7f25840cc5d0 0x7f25840c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 -- 172.21.15.33:0/3154499532 >> 172.21.15.33:0/3154499532 conn(0x7f258406eb10 msgr2=0x7f258406ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 -- 172.21.15.33:0/3154499532 shutdown_connections 2024-05-08T23:42:21.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.780+0000 7f2589cda700 1 -- 172.21.15.33:0/3154499532 wait complete. 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.781+0000 7f2589cda700 1 Processor -- start 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 -- start start 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25840cc5d0 0x7f2584156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 0x7f258406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840aafc0 con 0x7f25841576a0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840b2b50 con 0x7f25840cc5d0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.782+0000 7f2589cda700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584156180 con 0x7f25841572c0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38528/0 (socket says 172.21.15.33:38528) 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 -- 172.21.15.33:0/1470786450 learned_addr learned my addr 172.21.15.33:0/1470786450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2582ffd700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 0x7f258406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f25837fe700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25840cc5d0 0x7f2584156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 -- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 msgr2=0x7f258406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 0x7f258406c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 -- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25840cc5d0 msgr2=0x7f2584156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25840cc5d0 0x7f2584156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 -- 172.21.15.33:0/1470786450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009550 con 0x7f25841576a0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2582ffd700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 0x7f258406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2583fff700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f257c009d30 tx=0x7f257c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00cd40 con 0x7f25841576a0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f257c00f040 con 0x7f25841576a0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2584156b50 con 0x7f25841576a0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0146e0 con 0x7f25841576a0 2024-05-08T23:42:21.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.783+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841f05d0 con 0x7f25841576a0 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.784+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584075460 con 0x7f25841576a0 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.785+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f257c014840 con 0x7f25841576a0 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.785+0000 7f256bfff700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 0x7f257006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.785+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f257c08a020 con 0x7f25841576a0 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.785+0000 7f25837fe700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 0x7f257006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:21.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.786+0000 7f25837fe700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 0x7f257006d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2578009c10 tx=0x7f257801b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:21.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:21.790+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f257c055020 con 0x7f25841576a0 2024-05-08T23:42:21.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:21.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:22.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.057+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f258406f3e0 con 0x7f25841576a0 2024-05-08T23:42:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.058+0000 7f256bfff700 1 -- 172.21.15.33:0/1470786450 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7f257c018090 con 0x7f25841576a0 2024-05-08T23:42:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:42:22.054 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 msgr2=0x7f257006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 0x7f257006d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2578009c10 tx=0x7f257801b040 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 msgr2=0x7f25840b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f257c009d30 tx=0x7f257c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 shutdown_connections 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f257006ac90 0x7f257006d150 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25841576a0 0x7f25840b1720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25840cc5d0 0x7f2584156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 --2- 172.21.15.33:0/1470786450 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f25841572c0 0x7f258406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 >> 172.21.15.33:0/1470786450 conn(0x7f258406eb10 msgr2=0x7f258406dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 shutdown_connections 2024-05-08T23:42:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.060+0000 7f2589cda700 1 -- 172.21.15.33:0/1470786450 wait complete. 2024-05-08T23:42:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:42:22.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:42:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.212+0000 7f3fb1a93700 1 Processor -- start 2024-05-08T23:42:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.212+0000 7f3fb1a93700 1 -- start start 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fb1a93700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7f3fac154970 0x7f3fac06ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fb1a93700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fac0caad0 0x7f3fac15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fb1a93700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac15b350 con 0x7f3fac06b280 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fb1a93700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac15e8a0 con 0x7f3fac0caad0 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fb1a93700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac15a5d0 con 0x7f3fac154970 2024-05-08T23:42:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fabfff700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fac0caad0 0x7f3fac15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3faaffd700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3fac06b280 0x7f3fac0cb6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57140/0 (socket says 172.21.15.33:57140) 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3faaffd700 1 -- 172.21.15.33:0/4203812099 learned_addr learned my addr 172.21.15.33:0/4203812099 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.213+0000 7f3fab7fe700 1 --2- 172.21.15.33:0/4203812099 >> v2:172.21.15.42:3300/0 conn(0x7f3fac154970 0x7f3fac06ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3faa7fc700 1 -- 172.21.15.33:0/4203812099 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712113045 0 0) 0x7f3fac15b350 con 0x7f3fac06b280 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3faa7fc700 1 -- 172.21.15.33:0/4203812099 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f980035b0 con 0x7f3fac06b280 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3fabfff700 1 -- 172.21.15.33:0/4203812099 >> v2:172.21.15.42:3300/0 conn(0x7f3fac154970 msgr2=0x7f3fac06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3fabfff700 1 --2- 172.21.15.33:0/4203812099 >> v2:172.21.15.42:3300/0 conn(0x7f3fac154970 0x7f3fac06ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3fabfff700 1 -- 172.21.15.33:0/4203812099 >> v1:172.21.15.33:6789/0 conn(0x7f3fac06b280 legacy=0x7f3fac0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3fabfff700 1 -- 172.21.15.33:0/4203812099 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac15e5c0 con 0x7f3fac0caad0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3fabfff700 1 --2- 172.21.15.33:0/4203812099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac0caad0 0x7f3fac15d9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0d0d00 tx=0x7f3fa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724c330c8827e499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3faa7fc700 1 -- 172.21.15.33:0/4203812099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400e070 con 0x7f3fac0caad0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3faa7fc700 1 -- 172.21.15.33:0/4203812099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3fa400bb20 con 0x7f3fac0caad0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.214+0000 7f3faa7fc700 1 -- 172.21.15.33:0/4203812099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4012430 con 0x7f3fac0caad0 2024-05-08T23:42:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 -- 172.21.15.33:0/4203812099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac0caad0 msgr2=0x7f3fac15d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/4203812099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac0caad0 0x7f3fac15d9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0d0d00 tx=0x7f3fa400b330 comp rx=0 tx=0).stop 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 -- 172.21.15.33:0/4203812099 shutdown_connections 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/4203812099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac0caad0 0x7f3fac15d9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/4203812099 >> v2:172.21.15.42:3300/0 conn(0x7f3fac154970 0x7f3fac06ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 -- 172.21.15.33:0/4203812099 >> 172.21.15.33:0/4203812099 conn(0x7f3fac06cd80 msgr2=0x7f3fac06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 -- 172.21.15.33:0/4203812099 shutdown_connections 2024-05-08T23:42:22.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.215+0000 7f3fb1a93700 1 -- 172.21.15.33:0/4203812099 wait complete. 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.216+0000 7f3fb1a93700 1 Processor -- start 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.217+0000 7f3fb1a93700 1 -- start start 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3fac0caad0 0x7f3fac159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac154970 0x7f3fac14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac154da0 con 0x7f3fac06b280 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1f0c80 con 0x7f3fac154970 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fb1a93700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac157690 con 0x7f3fac0caad0 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38558/0 (socket says 172.21.15.33:38558) 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 -- 172.21.15.33:0/1003801174 learned_addr learned my addr 172.21.15.33:0/1003801174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fabfff700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac154970 0x7f3fac14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3faaffd700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3fac0caad0 0x7f3fac159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 -- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3fac0caad0 msgr2=0x7f3fac159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3fac0caad0 0x7f3fac159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 -- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac154970 msgr2=0x7f3fac14ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac154970 0x7f3fac14ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 -- 172.21.15.33:0/1003801174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4009a70 con 0x7f3fac06b280 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fab7fe700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e870 tx=0x7f3f9c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.218+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c009590 con 0x7f3fac06b280 2024-05-08T23:42:22.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.219+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3f9c013680 con 0x7f3fac06b280 2024-05-08T23:42:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.219+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac157930 con 0x7f3fac06b280 2024-05-08T23:42:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.219+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c013a10 con 0x7f3fac06b280 2024-05-08T23:42:22.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.219+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac154ff0 con 0x7f3fac06b280 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.220+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac157e50 con 0x7f3fac06b280 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.220+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3f9c013b70 con 0x7f3fac06b280 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.221+0000 7f3fa3fff700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 0x7f3f8c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.221+0000 7f3faaffd700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 0x7f3f8c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.221+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3f9c089340 con 0x7f3fac06b280 2024-05-08T23:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.221+0000 7f3faaffd700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 0x7f3f8c06d180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fac157530 tx=0x7f3f94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.225+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3f9c054330 con 0x7f3fac06b280 2024-05-08T23:42:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.493+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fac075460 con 0x7f3fac06b280 2024-05-08T23:42:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.493+0000 7f3fa3fff700 1 -- 172.21.15.33:0/1003801174 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6238 (secure 0 0 0) 0x7f3f9c056260 con 0x7f3fac06b280 2024-05-08T23:42:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:42:22.490 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/4270472911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6835","nonce":4270472911},{"type":"v1","addr":"172.21.15.33:6837","nonce":4270472911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4220,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1480873328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1480873328},{"type":"v1","addr":"172.21.15.33:6839","nonce":1480873328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2879762018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2879762018},{"type":"v1","addr":"172.21.15.42:6835","nonce":2879762018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/1012831721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":1012831721},{"type":"v1","addr":"172.21.15.42:6837","nonce":1012831721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6836/1676857648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1676857648},{"type":"v1","addr":"172.21.15.33:6836","nonce":1676857648}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:32:10.330935+0000","modified":"2024-05-08T23:32:16.353975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.42:6833/391209758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":391209758},{"type":"v1","addr":"172.21.15.42:6833","nonce":391209758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:42:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 msgr2=0x7f3f8c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 0x7f3f8c06d180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fac157530 tx=0x7f3f94009040 comp rx=0 tx=0).stop 2024-05-08T23:42:22.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 msgr2=0x7f3fac1557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e870 tx=0x7f3f9c00eb80 comp rx=0 tx=0).stop 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 shutdown_connections 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3f8c06acc0 0x7f3f8c06d180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fac154970 0x7f3fac14ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fac06b280 0x7f3fac1557f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 --2- 172.21.15.33:0/1003801174 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3fac0caad0 0x7f3fac159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 >> 172.21.15.33:0/1003801174 conn(0x7f3fac06cd80 msgr2=0x7f3fac14ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 shutdown_connections 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.496+0000 7f3fb1a93700 1 -- 172.21.15.33:0/1003801174 wait complete. 2024-05-08T23:42:22.493 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T23:42:22.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:42:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 Processor -- start 2024-05-08T23:42:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 -- start start 2024-05-08T23:42:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7fd9ac0c68e0 0x7fd9ac071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9ac0c6e30 0x7fd9ac071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ac1582e0 con 0x7fd9ac0c6310 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac157310 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.778+0000 7fd9b2390700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac157e50 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9ac0c6e30 0x7fd9ac071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd9ac0c6e30 0x7fd9ac071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38570/0 (socket says 172.21.15.33:38570) 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 -- 172.21.15.33:0/1666007384 learned_addr learned my addr 172.21.15.33:0/1666007384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9ab7fe700 1 --2- 172.21.15.33:0/1666007384 >> v2:172.21.15.42:3300/0 conn(0x7fd9ac0c68e0 0x7fd9ac071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 -- 172.21.15.33:0/1666007384 >> v2:172.21.15.42:3300/0 conn(0x7fd9ac0c68e0 msgr2=0x7fd9ac071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 --2- 172.21.15.33:0/1666007384 >> v2:172.21.15.42:3300/0 conn(0x7fd9ac0c68e0 0x7fd9ac071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 -- 172.21.15.33:0/1666007384 >> v1:172.21.15.42:6789/0 conn(0x7fd9ac0c6310 legacy=0x7fd9ac06baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 -- 172.21.15.33:0/1666007384 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0c66f0 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9b092d700 1 --2- 172.21.15.33:0/1666007384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c6e30 0x7fd9ac071c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ca290 tx=0x7fd9a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c668c5cef79f1bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9aaffd700 1 -- 172.21.15.33:0/1666007384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400e070 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9aaffd700 1 -- 172.21.15.33:0/1666007384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd9a400b910 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.779+0000 7fd9aaffd700 1 -- 172.21.15.33:0/1666007384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400bce0 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 -- 172.21.15.33:0/1666007384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c6e30 msgr2=0x7fd9ac071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1666007384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c6e30 0x7fd9ac071c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ca290 tx=0x7fd9a400b330 comp rx=0 tx=0).stop 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 -- 172.21.15.33:0/1666007384 shutdown_connections 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1666007384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c6e30 0x7fd9ac071c50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1666007384 >> v2:172.21.15.42:3300/0 conn(0x7fd9ac0c68e0 0x7fd9ac071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 -- 172.21.15.33:0/1666007384 >> 172.21.15.33:0/1666007384 conn(0x7fd9ac06d2b0 msgr2=0x7fd9ac06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 -- 172.21.15.33:0/1666007384 shutdown_connections 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.780+0000 7fd9b2390700 1 -- 172.21.15.33:0/1666007384 wait complete. 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.782+0000 7fd9b2390700 1 Processor -- start 2024-05-08T23:42:22.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.782+0000 7fd9b2390700 1 -- start start 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9ac0c6310 0x7fd9ac14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9ac0c6e30 0x7fd9ac1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac149760 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac152f10 con 0x7fd9ac0c6310 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b2390700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac151c80 con 0x7fd9ac0c6e30 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38578/0 (socket says 172.21.15.33:38578) 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9ab7fe700 1 -- 172.21.15.33:0/1787875945 learned_addr learned my addr 172.21.15.33:0/1787875945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9abfff700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9ac0c6310 0x7fd9ac14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.783+0000 7fd9b092d700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9ac0c6e30 0x7fd9ac1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 -- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9ac0c6e30 msgr2=0x7fd9ac1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9ac0c6e30 0x7fd9ac1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 -- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9ac0c6310 msgr2=0x7fd9ac14b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9ac0c6310 0x7fd9ac14b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 -- 172.21.15.33:0/1787875945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009b30 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9ab7fe700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000e9d0 tx=0x7fd9a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0009730 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd9a000ee70 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac14ef70 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000ca30 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.784+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac0c8570 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.785+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac152a00 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.786+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fd9a000cb90 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.786+0000 7fd9a8ff9700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 0x7fd99806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.786+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7fd9a0089480 con 0x7fd9ac0c68e0 2024-05-08T23:42:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.786+0000 7fd9abfff700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 0x7fd99806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.787+0000 7fd9abfff700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 0x7fd99806d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007fd0 tx=0x7fd99c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:22.791+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fd9ac152a00 con 0x7fd9ac0c68e0 2024-05-08T23:42:23.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.013+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9ac152a00 con 0x7fd9ac0c68e0 2024-05-08T23:42:23.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.015+0000 7fd9a8ff9700 1 -- 172.21.15.33:0/1787875945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16379 (secure 0 0 0) 0x7fd9ac152a00 con 0x7fd9ac0c68e0 2024-05-08T23:42:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T23:42:23.012 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"fsid":"7b439b97-64c0-4aab-ac3b-1b32ed376bfb","created":"2024-05-08T23:31:52.214077+0000","modified":"2024-05-08T23:32:59.845491+0000","last_up_change":"2024-05-08T23:32:03.174271+0000","last_in_change":"2024-05-08T23:31:55.047509+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:32:04.830340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:32:08.545052+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:32:09.301342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d43d2aca-6883-419c-8beb-8caa4f3e5453","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":47117},{"type":"v1","addr":"172.21.15.33:6801","nonce":47117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":47117},{"type":"v1","addr":"172.21.15.33:6803","nonce":47117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":47117},{"type":"v1","addr":"172.21.15.33:6807","nonce":47117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":47117},{"type":"v1","addr":"172.21.15.33:6805","nonce":47117}]},"public_addr":"172.21.15.33:6801/47117","cluster_addr":"172.21.15.33:6803/47117","heartbeat_back_addr":"172.21.15.33:6807/47117","heartbeat_front_addr":"172.21.15.33:6805/47117","state":["exists","up"]},{"osd":1,"uuid":"c8d8fcfd-af87-49c6-bc12-c769b062cb0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":47120},{"type":"v1","addr":"172.21.15.33:6817","nonce":47120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":47120},{"type":"v1","addr":"172.21.15.33:6819","nonce":47120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":47120},{"type":"v1","addr":"172.21.15.33:6823","nonce":47120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":47120},{"type":"v1","addr":"172.21.15.33:6821","nonce":47120}]},"public_addr":"172.21.15.33:6817/47120","cluster_addr":"172.21.15.33:6819/47120","heartbeat_back_addr":"172.21.15.33:6823/47120","heartbeat_front_addr":"172.21.15.33:6821/47120","state":["exists","up"]},{"osd":2,"uuid":"35734e60-7ecf-4a32-a934-4a9be2857016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":47118},{"type":"v1","addr":"172.21.15.33:6809","nonce":47118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":47118},{"type":"v1","addr":"172.21.15.33:6811","nonce":47118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":47118},{"type":"v1","addr":"172.21.15.33:6815","nonce":47118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":47118},{"type":"v1","addr":"172.21.15.33:6813","nonce":47118}]},"public_addr":"172.21.15.33:6809/47118","cluster_addr":"172.21.15.33:6811/47118","heartbeat_back_addr":"172.21.15.33:6815/47118","heartbeat_front_addr":"172.21.15.33:6813/47118","state":["exists","up"]},{"osd":3,"uuid":"b62de6c1-97fe-4574-9736-23c4c7a2ac56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":47140},{"type":"v1","addr":"172.21.15.33:6825","nonce":47140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":47140},{"type":"v1","addr":"172.21.15.33:6827","nonce":47140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":47140},{"type":"v1","addr":"172.21.15.33:6831","nonce":47140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":47140},{"type":"v1","addr":"172.21.15.33:6829","nonce":47140}]},"public_addr":"172.21.15.33:6825/47140","cluster_addr":"172.21.15.33:6827/47140","heartbeat_back_addr":"172.21.15.33:6831/47140","heartbeat_front_addr":"172.21.15.33:6829/47140","state":["exists","up"]},{"osd":4,"uuid":"081cd8bf-a960-4db7-9b4e-028fc22b43ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6823","nonce":47110},{"type":"v1","addr":"172.21.15.42:6825","nonce":47110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":47110},{"type":"v1","addr":"172.21.15.42:6827","nonce":47110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":47110},{"type":"v1","addr":"172.21.15.42:6831","nonce":47110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":47110},{"type":"v1","addr":"172.21.15.42:6829","nonce":47110}]},"public_addr":"172.21.15.42:6825/47110","cluster_addr":"172.21.15.42:6827/47110","heartbeat_back_addr":"172.21.15.42:6831/47110","heartbeat_front_addr":"172.21.15.42:6829/47110","state":["exists","up"]},{"osd":5,"uuid":"837c0f59-424f-4f17-8b56-e1e548e855f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":47100},{"type":"v1","addr":"172.21.15.42:6801","nonce":47100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":47100},{"type":"v1","addr":"172.21.15.42:6803","nonce":47100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":47100},{"type":"v1","addr":"172.21.15.42:6807","nonce":47100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":47100},{"type":"v1","addr":"172.21.15.42:6805","nonce":47100}]},"public_addr":"172.21.15.42:6801/47100","cluster_addr":"172.21.15.42:6803/47100","heartbeat_back_addr":"172.21.15.42:6807/47100","heartbeat_front_addr":"172.21.15.42:6805/47100","state":["exists","up"]},{"osd":6,"uuid":"e7c0c6d9-0fb2-44c8-90eb-59428c93016e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":47123},{"type":"v1","addr":"172.21.15.42:6817","nonce":47123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":47123},{"type":"v1","addr":"172.21.15.42:6819","nonce":47123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":47123},{"type":"v1","addr":"172.21.15.42:6824","nonce":47123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":47123},{"type":"v1","addr":"172.21.15.42:6821","nonce":47123}]},"public_addr":"172.21.15.42:6817/47123","cluster_addr":"172.21.15.42:6819/47123","heartbeat_back_addr":"172.21.15.42:6824/47123","heartbeat_front_addr":"172.21.15.42:6821/47123","state":["exists","up"]},{"osd":7,"uuid":"cf7a652a-2319-4560-b345-22bceefea886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":47101},{"type":"v1","addr":"172.21.15.42:6809","nonce":47101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":47101},{"type":"v1","addr":"172.21.15.42:6811","nonce":47101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":47101},{"type":"v1","addr":"172.21.15.42:6815","nonce":47101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":47101},{"type":"v1","addr":"172.21.15.42:6813","nonce":47101}]},"public_addr":"172.21.15.42:6809/47101","cluster_addr":"172.21.15.42:6811/47101","heartbeat_back_addr":"172.21.15.42:6815/47101","heartbeat_front_addr":"172.21.15.42:6813/47101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206137+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.190014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.206288+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.234052+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.204107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:31:59.960381+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.253723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:32:00.219917+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:42:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 msgr2=0x7fd99806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 0x7fd99806d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007fd0 tx=0x7fd99c009040 comp rx=0 tx=0).stop 2024-05-08T23:42:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 msgr2=0x7fd9ac14b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000e9d0 tx=0x7fd9a000ece0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 shutdown_connections 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fd99806ac90 0x7fd99806d150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fd9ac0c6e30 0x7fd9ac1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9ac0c6310 0x7fd9ac14b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 --2- 172.21.15.33:0/1787875945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9ac0c68e0 0x7fd9ac14b760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 >> 172.21.15.33:0/1787875945 conn(0x7fd9ac06d2b0 msgr2=0x7fd9ac072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:23.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 shutdown_connections 2024-05-08T23:42:23.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.017+0000 7fd9b2390700 1 -- 172.21.15.33:0/1787875945 wait complete. 2024-05-08T23:42:23.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:42:23.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.170+0000 7f3e0bfd5700 1 Processor -- start 2024-05-08T23:42:23.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.170+0000 7f3e0bfd5700 1 -- start start 2024-05-08T23:42:23.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0bfd5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040c6e30 0x7f3e0415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0bfd5700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e040c7210 0x7f3e04153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0bfd5700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0415e750 con 0x7f3e040668c0 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0bfd5700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e040709a0 con 0x7f3e040c7210 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0bfd5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0415b150 con 0x7f3e040c6e30 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e09570700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3e040c7210 0x7f3e04153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0a572700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3e040668c0 0x7f3e04070e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57144/0 (socket says 172.21.15.33:57144) 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.171+0000 7f3e0a572700 1 -- 172.21.15.33:0/4210369767 learned_addr learned my addr 172.21.15.33:0/4210369767 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09d71700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040c6e30 0x7f3e0415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09570700 1 -- 172.21.15.33:0/4210369767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040c6e30 msgr2=0x7f3e0415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09570700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040c6e30 0x7f3e0415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09570700 1 -- 172.21.15.33:0/4210369767 >> v1:172.21.15.33:6789/0 conn(0x7f3e040668c0 legacy=0x7f3e04070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09570700 1 -- 172.21.15.33:0/4210369767 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0415dbd0 con 0x7f3e040c7210 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e08d6f700 1 -- 172.21.15.33:0/4210369767 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292949790 0 0) 0x7f3e0415e750 con 0x7f3e040668c0 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e09570700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e04153d30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009840 tx=0x7f3e00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2efd437d3012b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e08d6f700 1 -- 172.21.15.33:0/4210369767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00011830 con 0x7f3e040c7210 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e08d6f700 1 -- 172.21.15.33:0/4210369767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3e00011990 con 0x7f3e040c7210 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.172+0000 7f3e08d6f700 1 -- 172.21.15.33:0/4210369767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00011d40 con 0x7f3e040c7210 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4210369767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 msgr2=0x7f3e04153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e04153d30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009840 tx=0x7f3e00009b50 comp rx=0 tx=0).stop 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4210369767 shutdown_connections 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040c6e30 0x7f3e0415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4210369767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e04153d30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4210369767 >> 172.21.15.33:0/4210369767 conn(0x7f3e0406a990 msgr2=0x7f3e0406ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4210369767 shutdown_connections 2024-05-08T23:42:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.173+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4210369767 wait complete. 2024-05-08T23:42:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.174+0000 7f3e0bfd5700 1 Processor -- start 2024-05-08T23:42:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.174+0000 7f3e0bfd5700 1 -- start start 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040668c0 0x7f3e040c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040c6e30 0x7f3e04151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040ccb20 con 0x7f3e040c7210 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040cd030 con 0x7f3e040c6e30 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0bfd5700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040ccd50 con 0x7f3e040668c0 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0a572700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0a572700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38598/0 (socket says 172.21.15.33:38598) 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e0a572700 1 -- 172.21.15.33:0/4141089422 learned_addr learned my addr 172.21.15.33:0/4141089422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e09d71700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040668c0 0x7f3e040c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.175+0000 7f3e09570700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040c6e30 0x7f3e04151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 -- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040668c0 msgr2=0x7f3e040c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040668c0 0x7f3e040c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 -- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040c6e30 msgr2=0x7f3e04151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040c6e30 0x7f3e04151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 -- 172.21.15.33:0/4141089422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e000094a0 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0a572700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc002230 tx=0x7f3dfc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00aac0 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3dfc00ac40 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0070d0 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0414cba0 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.176+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0414d160 con 0x7f3e040c7210 2024-05-08T23:42:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.177+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04075460 con 0x7f3e040c7210 2024-05-08T23:42:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.177+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f3dfc007230 con 0x7f3e040c7210 2024-05-08T23:42:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.178+0000 7f3df67fc700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 0x7f3de806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.178+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5912+0+0 (secure 0 0 0) 0x7f3dfc088f50 con 0x7f3e040c7210 2024-05-08T23:42:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.178+0000 7f3e09d71700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 0x7f3de806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.179+0000 7f3e09d71700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 0x7f3de806d180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006b20 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.183+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f3dfc053f40 con 0x7f3e040c7210 2024-05-08T23:42:23.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.449+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e040c94f0 con 0x7f3e040c7210 2024-05-08T23:42:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.458+0000 7f3df67fc700 1 -- 172.21.15.33:0/4141089422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3dfc018070 con 0x7f3e040c7210 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 msgr2=0x7f3de806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 0x7f3de806d180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006b20 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 msgr2=0x7f3e041f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc002230 tx=0x7f3dfc00cf50 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 shutdown_connections 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f3de806acc0 0x7f3de806d180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e040c7210 0x7f3e041f3550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f3e040668c0 0x7f3e040c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.460+0000 7f3e0bfd5700 1 --2- 172.21.15.33:0/4141089422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e040c6e30 0x7f3e04151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.461+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 >> 172.21.15.33:0/4141089422 conn(0x7f3e0406a990 msgr2=0x7f3e040c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.461+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 shutdown_connections 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.461+0000 7f3e0bfd5700 1 -- 172.21.15.33:0/4141089422 wait complete. 2024-05-08T23:42:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:42:23.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:42:23.498 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-05-08T23:42:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.616+0000 7fcb603e3700 1 Processor -- start 2024-05-08T23:42:23.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.616+0000 7fcb603e3700 1 -- start start 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb603e3700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb580c85a0 0x7fcb580c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb603e3700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb58070450 con 0x7fcb58152350 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb603e3700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb58070730 con 0x7fcb580c38a0 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb603e3700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb580c2e90 con 0x7fcb580c85a0 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e980700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb580c85a0 0x7fcb580c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e17f700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcb58152350 0x7fcb58158110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57154/0 (socket says 172.21.15.33:57154) 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e17f700 1 -- 172.21.15.33:0/704274872 learned_addr learned my addr 172.21.15.33:0/704274872 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e980700 1 -- 172.21.15.33:0/704274872 >> v1:172.21.15.42:6789/0 conn(0x7fcb580c38a0 legacy=0x7fcb580c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e980700 1 -- 172.21.15.33:0/704274872 >> v1:172.21.15.33:6789/0 conn(0x7fcb58152350 legacy=0x7fcb58158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5e980700 1 -- 172.21.15.33:0/704274872 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58070a10 con 0x7fcb580c85a0 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.617+0000 7fcb5d17d700 1 -- 172.21.15.33:0/704274872 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234042699 0 0) 0x7fcb58070450 con 0x7fcb58152350 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb5e980700 1 --2- 172.21.15.33:0/704274872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb580c85a0 0x7fcb580c8a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009d90 tx=0x7fcb5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbe232938f52b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb5d17d700 1 -- 172.21.15.33:0/704274872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000e070 con 0x7fcb580c85a0 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb5d17d700 1 -- 172.21.15.33:0/704274872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcb5000bbd0 con 0x7fcb580c85a0 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb5d17d700 1 -- 172.21.15.33:0/704274872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50012430 con 0x7fcb580c85a0 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb603e3700 1 -- 172.21.15.33:0/704274872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb580c85a0 msgr2=0x7fcb580c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb603e3700 1 --2- 172.21.15.33:0/704274872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb580c85a0 0x7fcb580c8a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009d90 tx=0x7fcb5000b330 comp rx=0 tx=0).stop 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb603e3700 1 -- 172.21.15.33:0/704274872 shutdown_connections 2024-05-08T23:42:23.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb603e3700 1 --2- 172.21.15.33:0/704274872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb580c85a0 0x7fcb580c8a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.618+0000 7fcb603e3700 1 -- 172.21.15.33:0/704274872 >> 172.21.15.33:0/704274872 conn(0x7fcb580666c0 msgr2=0x7fcb58066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.619+0000 7fcb603e3700 1 -- 172.21.15.33:0/704274872 shutdown_connections 2024-05-08T23:42:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.619+0000 7fcb603e3700 1 -- 172.21.15.33:0/704274872 wait complete. 2024-05-08T23:42:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.620+0000 7fcb603e3700 1 Processor -- start 2024-05-08T23:42:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.620+0000 7fcb603e3700 1 -- start start 2024-05-08T23:42:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb580c38a0 0x7fcb580c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb580c85a0 0x7fcb5806c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5806cab0 con 0x7fcb58152350 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580c6aa0 con 0x7fcb580c38a0 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb603e3700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580c5bf0 con 0x7fcb580c85a0 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38610/0 (socket says 172.21.15.33:38610) 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 -- 172.21.15.33:0/1021145152 learned_addr learned my addr 172.21.15.33:0/1021145152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e17f700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb580c38a0 0x7fcb580c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5d97e700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb580c85a0 0x7fcb5806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 -- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb580c85a0 msgr2=0x7fcb5806c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb580c85a0 0x7fcb5806c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 -- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb580c38a0 msgr2=0x7fcb580c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb580c38a0 0x7fcb580c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 -- 172.21.15.33:0/1021145152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50009a70 con 0x7fcb58152350 2024-05-08T23:42:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb5e980700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb50014040 tx=0x7fcb50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.621+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50012430 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.622+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcb50020070 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.622+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb580c5e90 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.622+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001bdf0 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.622+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb580c6450 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.623+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb581e25a0 con 0x7fcb58152350 2024-05-08T23:42:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.623+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7fcb50026400 con 0x7fcb58152350 2024-05-08T23:42:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.624+0000 7fcb4effd700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 0x7fcb3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.624+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5998+0+0 (secure 0 0 0) 0x7fcb5008b330 con 0x7fcb58152350 2024-05-08T23:42:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.624+0000 7fcb5e17f700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 0x7fcb3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.624+0000 7fcb5e17f700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 0x7fcb3c06d150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb54000f60 tx=0x7fcb54004a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.629+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7fcb581e25a0 con 0x7fcb58152350 2024-05-08T23:42:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:23.900+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb581e25a0 con 0x7fcb58152350 2024-05-08T23:42:24.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.463+0000 7fcb4effd700 1 -- 172.21.15.33:0/1021145152 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb581e25a0 con 0x7fcb58152350 2024-05-08T23:42:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 msgr2=0x7fcb3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 0x7fcb3c06d150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb54000f60 tx=0x7fcb54004a50 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 msgr2=0x7fcb5806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb50014040 tx=0x7fcb50004ad0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 shutdown_connections 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7fcb3c06ac90 0x7fcb3c06d150 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb58152350 0x7fcb5806c570 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb580c38a0 0x7fcb580c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 --2- 172.21.15.33:0/1021145152 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7fcb580c85a0 0x7fcb5806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 >> 172.21.15.33:0/1021145152 conn(0x7fcb580666c0 msgr2=0x7fcb5814f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 shutdown_connections 2024-05-08T23:42:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.466+0000 7fcb603e3700 1 -- 172.21.15.33:0/1021145152 wait complete. 2024-05-08T23:42:24.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:42:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.628+0000 7ff1fb74d700 1 Processor -- start 2024-05-08T23:42:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.628+0000 7ff1fb74d700 1 -- start start 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1fb74d700 1 --2- >> v2:172.21.15.42:3300/0 conn(0x7ff1f4154690 0x7ff1f406cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1fb74d700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1f40c2890 0x7ff1f406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1fb74d700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f415dcf0 con 0x7ff1f415d820 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1fb74d700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f40c2f20 con 0x7ff1f40c2890 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1fb74d700 1 -- --> v2:172.21.15.42:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f415a230 con 0x7ff1f4154690 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f9cea700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff1f40c2890 0x7ff1f406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f8ce8700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1f415d820 0x7ff1f415a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:57170/0 (socket says 172.21.15.33:57170) 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f8ce8700 1 -- 172.21.15.33:0/874541049 learned_addr learned my addr 172.21.15.33:0/874541049 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f94e9700 1 --2- 172.21.15.33:0/874541049 >> v2:172.21.15.42:3300/0 conn(0x7ff1f4154690 0x7ff1f406cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f9cea700 1 -- 172.21.15.33:0/874541049 >> v2:172.21.15.42:3300/0 conn(0x7ff1f4154690 msgr2=0x7ff1f406cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f9cea700 1 --2- 172.21.15.33:0/874541049 >> v2:172.21.15.42:3300/0 conn(0x7ff1f4154690 0x7ff1f406cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f9cea700 1 -- 172.21.15.33:0/874541049 >> v1:172.21.15.33:6789/0 conn(0x7ff1f415d820 legacy=0x7ff1f415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1f9cea700 1 -- 172.21.15.33:0/874541049 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f415dfb0 con 0x7ff1f40c2890 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.629+0000 7ff1ebfff700 1 -- 172.21.15.33:0/874541049 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696209973 0 0) 0x7ff1f415dcf0 con 0x7ff1f415d820 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1f9cea700 1 --2- 172.21.15.33:0/874541049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f406cff0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4067790 tx=0x7ff1ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706cf2e6bfcc2757 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1ebfff700 1 -- 172.21.15.33:0/874541049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00e070 con 0x7ff1f40c2890 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1ebfff700 1 -- 172.21.15.33:0/874541049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff1ec00bb20 con 0x7ff1f40c2890 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1ebfff700 1 -- 172.21.15.33:0/874541049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec012430 con 0x7ff1f40c2890 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 -- 172.21.15.33:0/874541049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 msgr2=0x7ff1f406cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/874541049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f406cff0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4067790 tx=0x7ff1ec00b330 comp rx=0 tx=0).stop 2024-05-08T23:42:24.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 -- 172.21.15.33:0/874541049 shutdown_connections 2024-05-08T23:42:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/874541049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f406cff0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/874541049 >> v2:172.21.15.42:3300/0 conn(0x7ff1f4154690 0x7ff1f406cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.630+0000 7ff1fb74d700 1 -- 172.21.15.33:0/874541049 >> 172.21.15.33:0/874541049 conn(0x7ff1f406e890 msgr2=0x7ff1f406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.631+0000 7ff1fb74d700 1 -- 172.21.15.33:0/874541049 shutdown_connections 2024-05-08T23:42:24.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.631+0000 7ff1fb74d700 1 -- 172.21.15.33:0/874541049 wait complete. 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.632+0000 7ff1fb74d700 1 Processor -- start 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.632+0000 7ff1fb74d700 1 -- start start 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f4154690 0x7ff1f41537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1f415d820 0x7ff1f414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f415d370 con 0x7ff1f40c2890 2024-05-08T23:42:24.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41493c0 con 0x7ff1f4154690 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1fb74d700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41496a0 con 0x7ff1f415d820 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38628/0 (socket says 172.21.15.33:38628) 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 -- 172.21.15.33:0/2180628622 learned_addr learned my addr 172.21.15.33:0/2180628622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f8ce8700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f4154690 0x7ff1f41537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f9cea700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1f415d820 0x7ff1f414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 -- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1f415d820 msgr2=0x7ff1f414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1f415d820 0x7ff1f414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 -- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f4154690 msgr2=0x7ff1f41537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f4154690 0x7ff1f41537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 -- 172.21.15.33:0/2180628622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec009a70 con 0x7ff1f40c2890 2024-05-08T23:42:24.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.633+0000 7ff1f94e9700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000e9b0 tx=0x7ff1f000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.634+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0009710 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.634+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff1f000ee50 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.634+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000ca70 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.634+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f4149980 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.634+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f4153cf0 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.635+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f4149ea0 con 0x7ff1f40c2890 2024-05-08T23:42:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.635+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7ff1f000cbd0 con 0x7ff1f40c2890 2024-05-08T23:42:24.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.636+0000 7ff1e9ffb700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 0x7ff1d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:24.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.636+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5998+0+0 (secure 0 0 0) 0x7ff1f00894f0 con 0x7ff1f40c2890 2024-05-08T23:42:24.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.636+0000 7ff1f8ce8700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 0x7ff1d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:24.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.636+0000 7ff1f8ce8700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 0x7ff1d806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4145890 tx=0x7ff1e4007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:24.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.640+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7ff1f4149ea0 con 0x7ff1f40c2890 2024-05-08T23:42:24.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:24.859+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1f4149ea0 con 0x7ff1f40c2890 2024-05-08T23:42:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.638+0000 7ff1e9ffb700 1 -- 172.21.15.33:0/2180628622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff1f4149ea0 con 0x7ff1f40c2890 2024-05-08T23:42:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 msgr2=0x7ff1d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 0x7ff1d806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4145890 tx=0x7ff1e4007e70 comp rx=0 tx=0).stop 2024-05-08T23:42:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 msgr2=0x7ff1f4148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000e9b0 tx=0x7ff1f000ecc0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 shutdown_connections 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7ff1d806ac90 0x7ff1d806d150 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7ff1f415d820 0x7ff1f414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1f40c2890 0x7ff1f4148d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 --2- 172.21.15.33:0/2180628622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1f4154690 0x7ff1f41537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 >> 172.21.15.33:0/2180628622 conn(0x7ff1f406e890 msgr2=0x7ff1f406ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 shutdown_connections 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.644+0000 7ff1fb74d700 1 -- 172.21.15.33:0/2180628622 wait complete. 2024-05-08T23:42:25.641 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:42:25.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:42:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.804+0000 7f8306b8a700 1 Processor -- start 2024-05-08T23:42:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.804+0000 7f8306b8a700 1 -- start start 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8306b8a700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f830015a380 0x7f830006e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8306b8a700 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830006fe50 con 0x7f830015d970 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8306b8a700 1 -- --> v1:172.21.15.42:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8300154a90 con 0x7f83001546b0 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8306b8a700 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000709f0 con 0x7f830015a380 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8305127700 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f830015a380 0x7f830006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f82fffff700 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f830015d970 0x7f830006e060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:37656/0 (socket says 172.21.15.33:37656) 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f82fffff700 1 -- 172.21.15.33:0/2613609088 learned_addr learned my addr 172.21.15.33:0/2613609088 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8305127700 1 -- 172.21.15.33:0/2613609088 >> v1:172.21.15.42:6789/0 conn(0x7f83001546b0 legacy=0x7f83000702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8305127700 1 -- 172.21.15.33:0/2613609088 >> v1:172.21.15.33:6789/0 conn(0x7f830015d970 legacy=0x7f830006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f82ff7fe700 1 -- 172.21.15.33:0/2613609088 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829538341 0 0) 0x7f830006fe50 con 0x7f830015d970 2024-05-08T23:42:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.805+0000 7f8305127700 1 -- 172.21.15.33:0/2613609088 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830015a7d0 con 0x7f830015a380 2024-05-08T23:42:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8305127700 1 --2- 172.21.15.33:0/2613609088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015a380 0x7f830006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009d90 tx=0x7f82f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b2e70b51f51784 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f82ff7fe700 1 -- 172.21.15.33:0/2613609088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800e070 con 0x7f830015a380 2024-05-08T23:42:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f82ff7fe700 1 -- 172.21.15.33:0/2613609088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f82f800bbd0 con 0x7f830015a380 2024-05-08T23:42:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f82ff7fe700 1 -- 172.21.15.33:0/2613609088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8012430 con 0x7f830015a380 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 -- 172.21.15.33:0/2613609088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015a380 msgr2=0x7f830006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 --2- 172.21.15.33:0/2613609088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015a380 0x7f830006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009d90 tx=0x7f82f800b330 comp rx=0 tx=0).stop 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 -- 172.21.15.33:0/2613609088 shutdown_connections 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 --2- 172.21.15.33:0/2613609088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015a380 0x7f830006e770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 -- 172.21.15.33:0/2613609088 >> 172.21.15.33:0/2613609088 conn(0x7f83000cde70 msgr2=0x7f83000ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 -- 172.21.15.33:0/2613609088 shutdown_connections 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.806+0000 7f8306b8a700 1 -- 172.21.15.33:0/2613609088 wait complete. 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.808+0000 7f8306b8a700 1 Processor -- start 2024-05-08T23:42:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.808+0000 7f8306b8a700 1 -- start start 2024-05-08T23:42:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83001546b0 0x7f83001f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 --2- >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f830015a380 0x7f83000c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c76c0 con 0x7f830015d970 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c7030 con 0x7f83001546b0 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8306b8a700 1 -- --> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c60f0 con 0x7f830015a380 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35864/0 (socket says 172.21.15.33:35864) 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 -- 172.21.15.33:0/3738489207 learned_addr learned my addr 172.21.15.33:0/3738489207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8304926700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83001546b0 0x7f83001f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f82fffff700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f830015a380 0x7f83000c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 -- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f830015a380 msgr2=0x7f83000c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f830015a380 0x7f83000c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 -- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83001546b0 msgr2=0x7f83001f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83001546b0 0x7f83001f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 -- 172.21.15.33:0/3738489207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009a70 con 0x7f830015d970 2024-05-08T23:42:25.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f8305127700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82f8014040 tx=0x7f82f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:25.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.809+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8012430 con 0x7f830015d970 2024-05-08T23:42:25.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.810+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f82f8020070 con 0x7f830015d970 2024-05-08T23:42:25.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.810+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8012e10 con 0x7f830015d970 2024-05-08T23:42:25.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.810+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000c5e20 con 0x7f830015d970 2024-05-08T23:42:25.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.810+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000c6690 con 0x7f830015d970 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.811+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83000678c0 con 0x7f830015d970 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.811+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 89145+0+0 (secure 0 0 0) 0x7f82f8026400 con 0x7f830015d970 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.812+0000 7f82fd7fa700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 0x7f82e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.812+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5560+0+0 (secure 0 0 0) 0x7f82f808b330 con 0x7f830015d970 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.812+0000 7f8304926700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 0x7f82e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:42:25.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.812+0000 7f8304926700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 0x7f82e406d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000f60 tx=0x7f82f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:42:25.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:25.816+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182498 (secure 0 0 0) 0x7f82f8056510 con 0x7f830015d970 2024-05-08T23:42:26.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.041+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83000c7170 con 0x7f830015d970 2024-05-08T23:42:26.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.640+0000 7f82fd7fa700 1 -- 172.21.15.33:0/3738489207 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f83000c7170 con 0x7f830015d970 2024-05-08T23:42:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.694+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 msgr2=0x7f82e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.694+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 0x7f82e406d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000f60 tx=0x7f82f4009040 comp rx=0 tx=0).stop 2024-05-08T23:42:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.694+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 msgr2=0x7f830015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:42:26.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.694+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f82f8014040 tx=0x7f82f8004ad0 comp rx=0 tx=0).stop 2024-05-08T23:42:26.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 shutdown_connections 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:6832/46840,v1:172.21.15.33:6833/46840] conn(0x7f82e406ac90 0x7f82e406d150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015d970 0x7f830015c3f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83001546b0 0x7f83001f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 --2- 172.21.15.33:0/3738489207 >> [v2:172.21.15.42:3301/0,v1:172.21.15.42:6790/0] conn(0x7f830015a380 0x7f83000c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 >> 172.21.15.33:0/3738489207 conn(0x7f83000cde70 msgr2=0x7f8300158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 shutdown_connections 2024-05-08T23:42:26.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T23:42:26.706+0000 7f8306b8a700 1 -- 172.21.15.33:0/3738489207 wait complete. 2024-05-08T23:42:26.711 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-05-08T23:42:26.729 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T23:42:26.730 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T23:42:26.730 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.741 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T23:42:26.741 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T23:42:26.741 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.795 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T23:42:26.795 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-05-08T23:42:26.795 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.853 INFO:tasks.ceph.mds.e:Stopped 2024-05-08T23:42:26.853 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T23:42:26.853 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.864 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T23:42:26.865 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T23:42:26.865 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.916 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T23:42:26.917 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-08T23:42:26.917 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:26.969 INFO:tasks.ceph.mds.f:Stopped 2024-05-08T23:42:26.969 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T23:42:26.969 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T23:42:26.969 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.052 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T23:42:27.053 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T23:42:27.053 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.125 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T23:42:27.125 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T23:42:27.125 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.193 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T23:42:27.193 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T23:42:27.193 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.262 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T23:42:27.262 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T23:42:27.262 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.329 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T23:42:27.329 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T23:42:27.329 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.406 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T23:42:27.407 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T23:42:27.407 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.467 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T23:42:27.467 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T23:42:27.467 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.543 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T23:42:27.543 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T23:42:27.543 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T23:42:27.543 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.573 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T23:42:27.574 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T23:42:27.574 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.599 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T23:42:27.600 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T23:42:27.600 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T23:42:27.600 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.628 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T23:42:27.629 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T23:42:27.629 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.652 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T23:42:27.652 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T23:42:27.652 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-08T23:42:27.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-08T23:42:27.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-08T23:42:27.706 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T23:42:27.706 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T23:42:27.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T23:42:27.756 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:42:27.757 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T23:42:28.040 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:42:28.040 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T23:42:28.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:42:28.147 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T23:42:28.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:42:28.243 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T23:42:28.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:42:28.366 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T23:42:28.674 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:42:28.674 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T23:42:28.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:42:28.779 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T23:42:28.872 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:42:28.873 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T23:42:28.973 INFO:tasks.ceph:Archiving mon data... 2024-05-08T23:42:28.974 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/data/mon.a.tgz 2024-05-08T23:42:28.978 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-05-08T23:42:29.001 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.cIWC3RVZ64 2024-05-08T23:42:29.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cIWC3RVZ64 2024-05-08T23:42:29.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.cIWC3RVZ64 2024-05-08T23:42:29.260 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.cIWC3RVZ64 is 461KB 2024-05-08T23:42:29.335 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.cIWC3RVZ64 2024-05-08T23:42:29.357 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/data/mon.b.tgz 2024-05-08T23:42:29.357 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-05-08T23:42:29.379 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.59sTqLg9kG 2024-05-08T23:42:29.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.59sTqLg9kG 2024-05-08T23:42:29.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.59sTqLg9kG 2024-05-08T23:42:29.637 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.59sTqLg9kG is 458KB 2024-05-08T23:42:29.711 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.59sTqLg9kG 2024-05-08T23:42:29.732 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/data/mon.c.tgz 2024-05-08T23:42:29.733 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-05-08T23:42:29.794 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.FNSoPH8qUD 2024-05-08T23:42:29.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FNSoPH8qUD 2024-05-08T23:42:29.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.FNSoPH8qUD 2024-05-08T23:42:30.050 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.FNSoPH8qUD is 461KB 2024-05-08T23:42:30.123 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.FNSoPH8qUD 2024-05-08T23:42:30.144 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T23:42:30.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:42:30.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T23:42:30.218 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-05-08T23:42:30.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:42:30.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-05-08T23:42:30.255 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:42:30.294 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-05-08T23:42:30.302 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T23:42:30.304 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi033/crash 2024-05-08T23:42:30.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:42:30.339 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi042/crash 2024-05-08T23:42:30.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T23:42:30.376 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T23:42:30.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:30.384 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:30.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:30.515 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T23:42:30.549 INFO:tasks.ceph:Compressing logs... 2024-05-08T23:42:30.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:42:30.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-05-08T23:42:32.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-05-08T23:44:37.152 INFO:tasks.ceph:Archiving logs... 2024-05-08T23:44:37.152 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi033/log 2024-05-08T23:44:37.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:44:50.209 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi042/log 2024-05-08T23:44:50.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:45:04.590 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T23:45:04.601 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T23:45:04.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:45:04.604 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:45:04.696 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T23:45:04.705 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T23:45:04.706 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:45:04.708 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:45:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-08T23:45:04.728 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2024-05-08T23:45:04.732 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:45:04.732 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +1161us[+1168us] +/- 56ms 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -1065us[-1058us] +/- 64ms 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -289us[ -282us] +/- 79ms 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-05-08T23:45:04.733 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1156us[+1170us] +/- 56ms 2024-05-08T23:45:04.734 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -1075us[-1062us] +/- 64ms 2024-05-08T23:45:04.734 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -271us[ -271us] +/- 80ms 2024-05-08T23:45:04.734 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:45:04.735 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T23:45:04.749 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T23:45:04.750 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T23:45:04.791 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:45:04.876 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:45:04.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:45:04.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:45:04.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:45:05.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:45:05.029 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T23:45:05.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T23:45:05.083 INFO:teuthology.task.internal:Duration was 1691.118859 seconds 2024-05-08T23:45:05.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T23:45:05.132 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T23:45:05.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:45:05.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:45:05.193 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:45:05.195 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:45:05.583 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T23:45:05.583 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T23:45:05.584 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:45:05.611 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-05-08T23:45:05.612 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:45:05.640 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T23:45:05.640 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:45:05.655 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:45:05.734 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T23:45:05.734 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:45:05.737 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:45:05.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T23:45:05.857 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T23:45:05.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:45:05.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:45:05.921 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T23:45:05.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:45:05.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:45:05.964 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-05-08T23:45:05.995 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-05-08T23:45:06.018 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:45:06.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:45:06.043 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:45:06.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:45:06.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T23:45:06.099 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T23:45:06.101 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi033 2024-05-08T23:45:06.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:45:06.148 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698175/remote/smithi042 2024-05-08T23:45:06.148 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:45:06.193 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T23:45:06.194 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:45:06.196 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:45:06.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T23:45:06.261 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T23:45:06.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T23:45:06.300 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T23:45:06.301 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:45:06.303 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:45:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:45 /home/ubuntu/cephtest 2024-05-08T23:45:06.325 INFO:teuthology.orchestra.run.smithi042.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:45 /home/ubuntu/cephtest 2024-05-08T23:45:06.326 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T23:45:06.368 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T23:45:06.470 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} duration: 1691.1188588142395 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T23:45:06.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:45:06.552 INFO:teuthology.run:pass